I know it's the second to last talk and it's the last day of the conch here, so lunch is settling in to keep you engaged Before I get started, I don't know why more folks don't start with the thank-yous First I want to thank my wife and my children for letting me stay up past my bedtime to get things like this done And also the national area closure user group for fielding the first version of this talk There's a third group of folks I want to thank and it's a little more relevant to the talk here and we know the community is full of them And that is paradigm junkies I know the word junkie is a negative connotation in general, but you know closure itself as we know has quite the eclectic background pulling from rich Hickey's Amazon bookshelf and and Michael Fogus's and You know pulling from things to do and some things not to do from other languages And we've managed to come up with a pretty practical platform, right? I think that's part of the reason we're all here We enjoy paradigm shifts. We enjoy learning brand new things that also get things done For this particular talk through a couple of paradigm junkies. We want to thank in particular and that's these guys As we know so for yesterday Dan and will humbled us with quines twines thrines type inferences From any Cameron and David Nolan a community has ported that to closure The only thing you should infer from these photos is that Dan and will you need bigger pictures online that I can pull from I Looked all I could for both of you, but didn't happen So logic programming we've seen a couple of talks yesterday As I said, Dan and will give us a deep dive and earlier today We had a good talk on some of the high-level points in the history behind different parts of logic programming I want to establish a common vocabulary in a common language and really just center on the parts of core logic that I use for web testing So if you go to the core logic wiki, you'll see an example much like this where we're Putting out some facts and relationships and querying on it and I've tweaked a little bit So let's walk through this piece by piece. So they're all on the same page for how core logic works So when we first come to core logic, it's somewhat of a black box and it kind of should be even after you know How it works so it's more important than ever to keep in mind what we're responsible for in our logic program And what the engine is supposed to do for us. So let's step through those our first job is to Create a language that defines how we're going to relate things by default core logic has a few relations built in For lists and other primitive values, but how are we going to talk about the relationships in our application domain? So we start off with doing that and we have these deferral forms that provide for us kind of simple named relationships We're going to talk about men women fun people and people who like people Once we have that vocabulary in place We can establish the actual relationships that we're going to be dealing with and so in this case We're going to hard code some values here that we have some men women Mary and Lucy are both fun nobody else's and Bob likes Mary John likes Martha and Ricky likes Lucy Once we have both a vocabulary for defining relationships and the actual relationships to work with the data We need to do something with that and that's where the really interesting part of logic programming comes in and that is to establish Expectations and I use word expectations is kind of our first key cue over to kind of a testing vocabulary, right? Making assertions adding constraints. So at the bottom here, this run form is our query We've seen this through the other examples that Dan and will gave I'm gonna pull it up to the middle So folks in the back and see it better This is the really important part. I'm just walk through this piece by piece We start with the run form now run is our entry point into logic world outside of which we get to be enclosure inside of which We play by core logics rules We have a star on the end tells the logic engine to give us every possible answer every possible scenario that makes our assertions true Next in line is the queue and this is kind of a convention for having our escape hatch This is our exit point from the logic world whatever happens to get bound to queue Which is a fresh logic variable. It has no particular value at the beginning Whatever we happen to unify with queue or bind to queue in the process of our program is the final return value of the whole run form So that's how I get real values out of out of logic world back into closure world So going next in line here We're gonna set up some nicely named Fresh logic variables in the in this case We don't need to have an extra one, but it's nice to have one that makes sense for the domain We're testing so we're gonna create a new fresh variable person and we're gonna ask things about that person So first there's two assertions I want to make is this person is the person fun and is the person liked by Ricky and Because these phrases are side-by-side and there's no other control structures These are conjunctive such as an and relationship both must be true for the whole run to succeed Finally if those two things are gonna hold we can take the person that was found by that logic run and bind it To queue and again that becomes the return value of the entire run form back to our regular functional programming space And the only person who satisfies all those constraints in this way is Lucy When notice here important is that it's actually a sequence because we have zero one or more than one answers So Lucy is the single answer in this case So we've all done all that work. We had to set up relationships We had to actually you know define the language for them instantiate them and now we you know We had to run a query over them. What is the logic engine going to give us for free? First and most simply and there's a lot more it does than this But this these are the things I want to focus on for our testing purposes I should be able to verify facts that we've encoded in our in our little mini database of people so Here we have is Lucy fun and Q as we said is the return value, but it's bound to nothing in this case So it returns to get the output of a fresh logic variable all the underscore dot numbers We saw that yesterday with the quines just pages and pages of underscore dot number So here we have yes Lucy is fun, but no Martha Martha's no fun so verification of facts and Secondly and possibly more interesting is that it should be able to follow Implications and sometimes people call this making you making inferences that word inferences kind of loaded But it should be able to follow the implications of the relationships that we established in our application So in this case it should be able to work through the fact that the same person is both fun and like by Ricky So let's take a grossly over simplified, you know view of this We know that none of these guys are fun or like by Ricky So we're gonna start with the right answer and walk our way through under the covers core logic keeps You know what's called a substitution map that? Associates the fresh variables along the way with actual values and sees if everything works out in the end We're gonna do it highly simplistically by replacing the word person with the person. We're trying so Lucy is the candidate. Is she fun? Yes, she is Is she like by Ricky? Yes, she is and so finally since she passed through both those phases Q having been bound to nothing else as part of this run is still fresh That's still available to be bound to unified with and so Lucy comes out as the only answer to this puzzle So given all of those behaviors given all that kind of characteristics of logic programming Just a priori what would incline us to think that logic programming is even suited for testing? Why would it make an interesting part of our testing toolkit? I think there are a few areas that come to mind at least for me So the first thing we said about the logic engine was its responsibility is to do some verification of facts in our application domain And we use the word verify What's part of what we're doing with our testing is we're trying to verify that values are a certain a certain value in a Certain context and the question we're gonna ask ourselves is are we already doing some? Modicum of testing by having a relational paradigm as part of our coding. So simple examples to look at cons I know it's closure. Where's the conge? I'm sorry so state cons But cons a onto bcd. I get ABCd nothing surprising there How do we transform this from a regular function to a relational function to a goal? Well, they didn't will told us yesterday. There's two steps first step add an o to the end All right, that's the most important step and as Craig and Dara said if you use a logic program You should you pull in you should alias the logic portion as L. So you can say el conzo el texto el membro It's not entirely culturally sensitive, but it'll be good for good reading So here we have conzo. We don't do things we've added the oh, so it's just a convention So we can keep saying looking between regular functions and relational functions But we've also tacked on one extra parameter here and as was mentioned yesterday What cons would return as output? We're going to tack on as the last argument of this relation and the whole idea is that we're talking about a relationship between two inputs and an Output for the idea of conzing So here we have built in that if I run this so we have q which is the fresh variable conzing a onto bcd Q should be bound to the only possible answer to this and we get back a list of answers with a single answer the list ABCd so If we read a hard code that we can give verification that this behavior is true So, you know, we're already doing so if we're testing cons ourselves like we were writing our own list been testing our own cons We would check that this operation gives that output So there's not a meta question as want us to keep in mind as we work through this is what you know If we're already doing some testing then what's that next level of testing we're going to think about and get into Going beyond that there's some kind of relatively smaller wins that we get as we've seen from all the programs here They're both in this example and from Dan and will Logic program has a declarative syntax and that gives us a win We're trying to encode complex conditions and complex flows that we have a declarative syntax at hand to kind of work through those a little more cleanly Perhaps a subtler point and when we have our own applications and we're doing high-level testing So this is where the focus of this talk is web testing which is high level We're testing a lot of different conditions at once I might step through my application three steps and I want to stop and I want to see you know five different conditions are true 10 different conditions are true and If we're not in by default when we encode as ourselves if else's cons We're adding kind of a sequentialness to our testing. We work through each one in part and we kind of map out the branches ourselves, but As was mentioned before Parts of many Cameron is that better kind of like that Can parts of our program be ordered arbitrarily and for certain things they can't be but for certain simple things they can be So for our original example, it shouldn't matter whether or not, you know We checked that she was fun Lucy was fun first or that Ricky liked her or even if we just bound two fresh variables together Made them you know associated in that substitutions map It shouldn't matter because by the end all the associations will come to make you know the right answer And then finally and this is kind of a more this was the point that kind of spans beyond testing It's testing exploration So at least for myself when working with separate QA teams One of the things that QA guys do a lot better than I do is taking my application and finding different ways to get to The same answer I'm taking different navigational paths so an application or exploring different workflows Which often exposes a brittleness and edge cases that I didn't consider There's no reason why our testing tools get also kind of open the door to a little bit more exploration Especially while we're authoring our tests and so I just want to take you know the example we looked at Here we're looking at a query that's really a combination of a statement and a question and the statement is about Ricky and who he likes and the question is About this fun person who is also liked by Ricky and what would it take for us to transform this program from a kind of It's half statement half question to a full open-ended open-ended question Well, we just replaced the hard-coded value with one extra fresh variable and we're done nothing else structurally changes So here we have person one in person two as opposed to person and we find that lo and behold Lucy isn't the only fun person Like by somebody else. There's also Bob and Mary. So this is a simple example of transforming Hard-coded values statements and making them questions And we're going to find as we kind of explore using logic programs for things that maybe they weren't originally intended for It was we dial the knob between making statements and asking questions a lot of power comes out of that So This goes beyond testing though. I don't need to use this for testing on this showing us we had a custom domain We defined our own vocabulary for relationships And now we have this kind of custom query language for talking about fun people and people who like people and so I don't think this is a great revelation for those who are to do a lot of logic programming But if you've come to core logic and you say well, you know, why would I even bother or you've learned it? Okay, I can do the zero puzzle I can do Sudoku maybe from really good But what might use this for and kind of my more maybe enterprise or you know Genuinely fully practical application and that is that core logic among other things is a query platform So under the covers we already have a query language right in terms of the relations that are part of core logic itself But we can also build our own custom query languages using core logic as a platform for that given the semantics of core logic And under the covers of course many Cameron This is all it takes to transform What would otherwise seem an unrelated domain into a core logic searchable space and there's an example of this on the Core logic wiki of using datomic and bringing in the datomic datums into the ecosystem of a logic program So it's not that hard once you've you know read people's dissertations and listen to David Nolan's talks multiple times Even then it's still kind of magic, right? All right, so we have these characteristics that so April or we haven't done any coding yet We kind of think this is going to work out Maybe there's some things that are going to contribute to the testing story and that kind of core logic as a as a query platform Seems pretty interesting. So move forward, but what do we get? We talked about paradigm junkies at the beginning Where's the paradigm shift? I mean, where's that really big that big win and I'm going to argue that perhaps with using logic programming in our tests We're going to get to a higher and possibly correct in certain circumstances level of abstraction in our tests And we're going to be called to test more and I don't mean that test in terms of number of test cases But actually the nature of tests and what we choose to test So that's logic programming just the crust of mini cannon and for logic, but it's all we're going to use for this talk So web testing what do I mean by web testing for this particular talk? In case it wasn't clear I'm talking about functional testing I was supposed to load testing I'm a system level high level behavioral tests that touch the whole stack as opposed to a little unit tests or integration tests and The final criterion so that I can use the libraries that I want to use is that it should behave like real users when run as much as possible And with these criteria in place there's really one open-source tool that kind of treats this end-to-end and that is selenium web driver So for those who don't know selenium web driver is a suite of tools used for automating programmatically automating graphical browsers So Firefox Chrome IE opera Safari and as I say as a suite of tools There's things like a Firefox add-on for authoring tests There's a distributed version for running your tests across multiple computers with different configurations our focus here is the simple client API is that it exposes especially the Java one and Wrapping the Java is a library called CLJ web driver Which consistently have two API is kind of a low-level core API That's close to the Java metal if you will and then a higher-level taxi web driver taxi Sorry API that is a high-level more terse API for forgetting the same things done And so the basis of this talk is taking those that same set of functions for manipulating a browser that will use for high-level Testing in the first place and combining that with the semantics of core logic bringing it into the logical realm Making our taking some of those regular functions and adding O's everywhere and turning them into logical logic functions or relations or goals So we started with this program and it's somewhat clear how it takes these values And it might put it into a search tree of some kind and traverse that we don't really care That's the whole point What what's the what's the corollary for web testing? Oh, it's the web page. It's all that we have right So how is this going to match up with our we said we had certain responsibilities as authors of logic programs And there were certain things the engine gave us so how's that going to pair up with our other program and the web page? So first what's that language for talking about the relationships between values? What are the values going to be elements in the page browsers browser specifications? How do we relate those? Well, you get that for free right? I wrote those So that's the language. It's a work in progress, of course and contributions are welcome But this sort of gives us a base vocabulary for talking about a web page in a relational manner Next we actually have to have data right no good, but we've already done that work right? We're testing at this point. We have an application. It's full of data Right the web page is chock full of simple things like named attributes and then the nature of the DOM itself is available to us We have hierarchies and relationships that are encoded and available to us programmatically from Selenium and its APIs So there's a host of information there that we can pull from to do our testing All right, and finally establishing expectations. So what does it look like to write a logic program that uses web? Web API is like CLJ web driver. Well, let's start with a couple of ones straight from CLJ web driver I hope these just make natural intuitive sense if I ask for the attribute of an element say the class attribute I'll get the value of that if I ask for its tag I get the tag text text size width and height enabled is it enabled and then Child relationships, you know, does this is a child of the second element? So we said before what do we do first step is? From transforming this from a regular function to a relational function. What's the first step? Add the O's okay. That's mean you're done No, we have to add the O's and then we also tack in the return value of the particular function as the last parameter So now we're talking about you know, what is the relationship for a particular element? It's class attribute and the value of that class attribute and going down the list. So Let's move on from this. Okay. That's our part. We're writing What's it was going to look like when we actually put this to use with the engine? So let's verify a fact about the page We looked at the GitHub login page before the home page of GitHub when you're not logged in it better have a login link, right? Or otherwise not getting in so let's make sure that has a log has a an anchor tag that points to that particular URL So start with run have our qsk patch set up a fresh variable L That's going to be our element. We're talking about Does the element have an a tag? Okay, it does move on and then is the href attribute of that anchor tag GitHub comm so this is kind of a you know a simple way to verify is that thing on the page and lo and behold It went and found it and we have the you know that logic variable output for q That indeed this is on the page So we have an a tag and it has that particular href if I were to change the a tag and ask for a block quote with that We're not going to find a block quote with that and we get the empty list for there are no answers to this particular query All right, so that was verification simple enough. I could have faked that out, right? It's not live coding How about following those implications? So let's take a slightly more complex example We're gonna start here again with q you have two fresh variables So I'm interested in finding a child element of another element on the page So we have L for the child element and parent L for the for the parent So the parent is going to have an idea of header So we're talking about some header div on the top of the page on GitHub We're talking in the neck the child element itself will have an attribute a class attribute of top nav So we're looking for some kind of menu perhaps on the top of the page And then I want to verify is that element in fact the child of its parent and at the end I'm only interested in seeing what that child element actually is so bind that to q and we'll get the answer And I think it deserves a stop here and look what we've done We've now managed to take a whole domain that value see in the bottom commented there is a value from CLJ Web driver representing an element on the page has a ul tag That's the top list of sign up for you for free explore everything else the top of github And we can take this value and pipe it right back into CLJ web driver We could click it we could flash it to see if it's actually there You know it's we've went from closure into the logic world. We did things according to its terms We got back real values So this is extending core logic and it's not it's not rocket science So let's test an actual app or a keynote app. This is the next stage of web development so we have a menu and We have an element with an idea of foo inside of that and I want to test the behavior that I come to this page And I click on foo and lights up and under the covers We're doing that by adding you know in a class of active to that particular element So a very simple behavior Hope we can keep it in our minds as we go through the next code examples We're going to test this twice. We're going to do once just kind of a straightforward CLJ web driver closure tools type of approach And then we'll try to see what happens if we use in logic programming and see what will possible effects that we get So we're going to start by just finding the element on the page CLJ web driver offers us this nice way to just pass in CSS queries and find things So we're looking for the first li under that particular ul. I grabbed that guy I'm going to pull out his attribute and everything was successful. What should the value of the class attribute be? Active okay, so we're going to test for that Using the is macro from closure dot test and we test that is that attribute in fact the string active Okay, we kind of cheated though right because if I were just to test that I clicked the first and only the first head active There's never a situation in a web app when like you click on something and multiple events fire by accident or just like lingering event Handlers or objects and it never happens in web development. So no, we needed to be a little more a little more robust here so let's go ahead and look at all the elements all the list items and To do that we're going to have to take go from element to elements and deal with a collection And so there's multiple ways to skin this closure collection cat But we're going to do it with reduce and we're just going to accept That's the right way at this point So if I were to work through that list of elements checking each one Do you have the right class and if you do I'm going to take your ID and toss it in a list a toss it in a vector and at the end at the end I'll get back, you know the sequence of All the elements that had a class of active and so if this were successful according to our current paradigm We'd get back a sequence of the single item the string foo for the element that had the class active And so we can test that like this Pretty straightforward. Maybe I want to test a multi-select behavior I can click multiple things and they're all lit up And maybe I'm gonna make a simpler collection handling here by using filter for example So I'll filter on elements that have the class attribute of active. I can just count them. I'm okay with that So that was a pretty straightforward, you know only a couple of functions from the CLG webdiver API and some standard closure A collection handling and testing. What's it look like when we're using logic programming? Let's just let's kind of explore Well first a pretty straight translation from the functional approach to the logic programming approach here So so grab that first element. We're going to unify him with q up front Okay, and then we'll just verify that everything we expect is true So does that element have a class attribute that is active now? If this comes back as a successful run if q returns to the element that we're interested in then we know We've won. We know that this has been verified. So this is kind of the first example We've already done some verification. Where's the testing going to come in? Where's the is macro? But we're not quite satisfied like before rock and test just the first element on test all the elements But before we jump into trying to think about closure collection handling here Let's think what we're trying to encode with this particular line We're trying to encode them looking for an element that happens to be an li and it's under this ul So there's a relationship there that here. I've captured as a CSS query. It's inside a string Perhaps a more proper place would be as a constraint in my logic program So this li thing should be just an element that has a particular tag It happens to live under this other element the ul So all of a sudden I now have because I'm using run star and asking for all the answers It's going to work through all those elements So I get the collection handling for free if you will due to the semantics of the logic run So if I take this guy and then we want to test that say foo was the only one clicked I'll just add one more constraint that indeed the ID of the element we're talking about is foo But as we said before we're still we're verifying the behavior, but where is the actual test? Where is the closure dot test is macro? There's a couple ways we could scan this right we could we could instead of having active and foo hard-coded here We could add more fresh variables and get them kind of pulled out of the run and look at those values like we did before You know is it a not is it a list of a single foo ID? Is it a is it three actives? But let's keep it hard-coded so we said before we're going to explore the difference between it You're making statements and asking open-ended questions here. We're gonna make a big statement You know here's all the things that this program should have they should be true about and we're going to test and see if this is true So if this is true, we said we're going to get back one value that that q should be one element So we could test that the run returns a single item in the collection of answers But that's really not what we're trying to test We're not trying to test that they got back a data structure with a single element We're trying to test if this logic run was deterministic if it had a single answer to the query and so it's a slightly different semantic and so we can go ahead and make some helper macros that wrap is and Include the idea of this is a deterministic relationship that succeeds just one time So let's go ahead and look at that So if we add s which is a macro available in web-driven logic, it says this run should succeed once and only once and Again, it's the same as testing if the collection is a single item the and the collection of answers But the idea is we're testing for a deterministic relationship and it's it's one success Maybe I'm testing the multi-select behavior I mentioned before and I can click multiple things and have them all active at the same time I can take out the ID as foo Constraint and say okay, it should succeed twice and only twice still deterministic, but multiple successes, right?  I have the same semantics here, but I can focus on what is the nature of the success of this logic run? and it's I'm somewhat of a nuanced point, but it's you know, I think it's a Different way to look at it and it's interesting that we can encapsulate all the Relationships that we're interested in in the logic program in the relational paradigm We're already working in and then focus on the success is this enough? No, it's not enough we have to eventually pull out the actual values We're testing sometimes and make sure that for example foo is the only one But once we're thinking about this in terms of the nature of the success of the logic run We can work with it differently when I'm not thinking about doing Collection handling as much as okay, this should succeed and the answer should be exactly foo or Maybe there's multiple answers and I clicked on multiple things and so at least in the answers There should be the items foo and bar that were clicked Or maybe I can just pass in a random predicate that takes in the answer as the first argument and Returns a true or a truthy or a falsi value and that dictates the nature of success so I The combination of things here the focus is We're at a higher level of abstraction We start putting all of our eggs in the logic program or as many as we need to and then And then thinking of for logic as more than just yes at the tool for solver in the zebra puzzle But also as really a platform for writing queries Focused on our own applications so Is it all fun and games? No, it's not this is one of the few times you're gonna actually use keynotes built in graphics. It's great So what are the risks and the first one's a human factor that I think is just worth mentioning pink elephant in the room At least in my experience Most QA folk are not hardcore Linux kernel hackers or working on the closure compiler if you are I'm not talking about you But there's generally a gulf between our QA teams if they're separate and developers and even wider gulf between those enclosure developers And there's an abyss between those and doing logic programming closure development, right? There's a big gap there. We're not talking about this we're talking about teams that are you know Companies that keep developers as their QA and simply switch up teams to test each other's code Or companies that have QA with paired engineers, so this is a human problem a half problem But I think just mention the obvious here cover of the magazine All right performance. This is the really interesting one, right? This is actually perform as the famous as Alan Pearl is quote that lists Programmers know the value of everything but the cost of nothing. I think Richieke has a version of that Let's be honest. There's performance issues up front before we think through the problem So when you're using CLJ web driver or really any conscientious closure API you get laziness by default Right if you ask for 2,000 elements off the page You're not going to get them unless you ask for all of them explicitly You get back a lazy seek of all the elements that match your particular query for example But we're using the logic engine is going to search the page for us if you don't give enough information It's got to go find it right so to search the whole DOM that's an expensive operation to traverse all of that Especially in selenium web driver So we have to think about the generality of our queries that we're passing in they don't ask for every anchor tag If you don't need every anchor tag We don't have any more laziness and of course the page size, but it doesn't take a very big page for this to get expensive in selenium web driver So what's the solution to this particular problem? How can we you know not incur this cost of searching this large space? With possibly expensive API calls. Well the solution that I've added in is poor man's constraints I say it's poor man's because I haven't read the papers that David is now reading on constraints So this is a way to constrain the logic engine to a portion of the page before you ever enter into your run form I provide a facility in web driver logic to dynamically rebind how much of the page gets searched for a particular run outside of the run I mean this gives us you know to sit tell gives blinders to the engine only look at all the dibs or only the header or only the content area Or only the anchor tags And then secondly kind of two layers of control both queries that are top-down and queries that are child or relationships And if you use the library that the tool makes sense as to why they're both there That's these guys So you'll see in early examples in the read me for this for the repo that these guys get used early because oftentimes It's expensive to search a large page for things or to let the logic engine do the searching for you The second side of that coin as I kind of hinted to is that selenium web driver is a factor of the problem I don't say it's a problem because there's so much excellent work put into the project But there are architectural factors that make searching the whole dom via selenium web driver an expensive operation For those who don't know under the covers selenium web driver consists of itself internally of a server and a client in the Server for chrome boots up jetty and it's controls the browser via a driver And your client code that you wrote in Java or the CLG web driver under the covers You think you're making function calls or method calls? No, no You're sitting HTTP requests to this server for everything you do and this is a I mean obviously an expensive operation and working on some kind of Internal closure data structure using regular functions And because of this caching is also a relatively small win for selenium web driver in particular You can hold on to a reference of the element you're talking about the actual Java object But every time you ask for its attribute or do something with it is going to be another call over HTTP to that server So it's a it's a win, but it's a small win and I'm already doing it and seal jweb driver So to be clear queries themselves are fast, you know It's it that the CSS queries you provide or parsed quickly and if you're finding one thing It's no big deal. It's when you start pulling in all the elements and instantiating them as part of that query That becomes a real issue. So what's the way to solve this? Well, there's no way except to get out of selenium web driver So I offer on a secondary namespace in this project called raw and all it does is take the page source from selenium It says thank you and tosses it into in live And now we have a regular set of closure maps with all the all the beauty and the performance of that and the in live selector syntax and there's a drastic I mean unspeakable performance improvement when you use this method So so such a big difference. I have to go back and look as to why CLG web driver has gotten slower And again Hugo Duncan. Thank you for a criterion. I caught that a couple months ago, and it's been great But what do we lose we lose the whole reason we started with selenium web driver Which is it gives us certain guarantees and if we haven't thought through the problem those might not be apparent But if you as a tester are writing this test sequence and you asked to click on an element And it's there in the DOM But there's another div that's come over it in time or it's styled in a way that it's off the page or there's JavaScript preventing you then you shouldn't be able to do that once we get out of selenium web driver We're looking at just the DOM structure. We no longer have those assurances that selenium gives us So if all you need is to verify the DOM, maybe you don't even need to be using selenium web driver in the first place But if you have to for authentication purposes or whatever else you can step out of it for a minute and do things in closure So Is this all we can do as I mentioned before we're using just the thin upper crust of core logic and mini-canon There's almost no it didn't see a condi up there. I can't even do an ore yet, right? Of course you can but it's not included here so the first thing is to know think through how can we fully leverage the mini-canon feature set and And by extension core logic. There's a lot more there that can go in and thinking about testing in the large and using our logic programs in different ways We've explored here just a few of the features there I encourage everyone to go read these things the reason schemer William Burr's dissertation I tried I couldn't finish it and this daunting quotation from Jim Zooey who said at that strange loop that you don't understand Mini-canon until you've implemented it yourself. I Hope that's not true Because I'm not gonna get there anytime soon, but It's it's a point well taken. There's a lot of subtlety there. It's not as simple as we might even think just talking about it So, please go read it. It's fun So in the last thing I want to show and good. I have plenty of time here Is if we take we said before an interesting way to think it's a kind of consider using logic programming for Our tests is to kind of encapsulate all the relational parts of our testing all the information That's related to each other in the logic program and to test in the nature of the success of that logic run But if we do that and the whole thing fails what part of that didn't pan out I had like 10 assertions which assertion was the one that didn't didn't work for me And so this might be a situation where this is definitely a situation where we need better Granule-level reporting and I think I have a couple of ideas of where this is going to go It's not there yet, but I think it's going to involve some deeper diving into into mini-canon itself into core logic But the last thing I want to show so every talk on core logic, you know Thank You Dan and will has to have the part where the program goes backwards and forwards it why So I realized this a couple weeks ago. I was like mine doesn't go backwards and forwards And this to a certain extent right so if you we asked to verify I found a particular element and I checked its class attribute Okay, that was you know I was going forwards and then I just gave the attributes and I found an element that satisfied that that was going backwards But they're not quite backwards. What is the quine of web testing with logic programming? I had to find it so I have a to work in progress Pre-pre-alpha, but when we do arithmetic using closure testing We do something like this five is not equal four and we get this nice output It tells us it failed and what you gave it and what you were supposed to have right and it actually evaluated I mean this there's there's the five what on earth would this look like for doing web testing, okay? So you did something that wasn't the right attribute. Maybe it was misspelled What are you gonna get back from that output? And so I thought through and thought maybe wouldn't it be great if you're running a test on an application And you know your test run failed and I was able to go okay I'm gonna show you what you had DOM wise and what you ought to have had for it to pass So I can give what you had we're testing what you already had How do I generate for you out of logic constraints the DOM needed that you needed to satisfy the constraints you specified in a logic run? So let's look at that This is the one life coding part This is what you were encountering wasn't it? There we go So We're gonna think about this in terms of closure data structures, right? So let's create an element that looks somewhat like what we'd see from it in it bigger, please Okay, I did a test earlier People said it was big enough All right, let's go here No worries Let's see if we get big enough yay nay More okay, okay. I was ready for this. Sorry Yay. All right. There's not very much code I'll keep that for later. Let's make this the size of my screen so Let's define a tag that looks like what we'd see if we're working within live It's a map form I has a tag It has some content and it has adders and we're going to ignore the fact that adders is a not a map right now It's okay implementation specific. This is the this is the client of web testing, right? So I have that guy I can then do the things we've been doing all along So I'm going to ask things about that data structure Good at you do and so I'll keep that guy in there So we're going to verify things about him is his class active and we should get back Can't see the output. That's good. Yes, it is. All right If we were to take out some of these values and ask questions about them So let's keep class and let's ask what is act and what is the value of that particular attribute? We're going to see active. Okay, that still works and if we go the other way and we ask for What is active? Well, the class attribute is active So if I were to then go one step further and say, okay, we're just take this guy out I want you to find me the answer For the purposes of everything else I demonstrated here We'd go to the web page we'd find we let it search the DOM and find an element that had a class attribute That was active, but I'm saying now we want to generate we're going to create DOM for you On the fly. That's what does that look like? That kind of looks like the data structure we specified above I only gave you a couple of the things to verify that the class was active I gave you a default tag and it's a short hop skip and a jump to go from something like that To something like that where you're working on that something you found a div You thought it had the right thing Here's that you needed and the DOM on the right the markup on the right was generated on the fly Using just the constraints in your logic program. So this is kind of where I see different directions He's going. Thank you The quine of web testing. Anyway, that's all I have. Thank you very much Looks like we have five minutes so any questions Yes, yes Simple for me was I was simply wrapping the is macro. I'm sorry. Yes, of course The question was I introduced a couple of handy macros like s and s plus and s as etc. That are macros and the question was why weren't they functions and The reason was that I was looking to simply wrap the is macro And so I built a macro to then you know delay the evaluation to things that get past to is so Is as I said before that's right is s and its friends add kind of an extra level of testing that you had One value that it was between this range that included these items. So it's really just to wrap closure test is the question Yeah, the questions. Yes Sure So the raw namespace that I mentioned actually works within live and in live works with closure maps So with closure maps are the question was Can this be extended to say JSON formatted data or XML and the answer is yes That you know when we already have tools in the closure community for transforming XML into map structures and to kind of traverse over them Whether it's zippers or other kind of declarative things that have been built And so we can definitely take those same semantics and I think we've seen some work with maps and core logic here with another presentation Yes, absolutely in the back in the logic program itself You're saying the question is can we the common the common idiom of having kind of before and after Things happen around test cases and test runs can we encode that in a logic program? And I'll be honest that I envision this part being the part where you've you're kind of done the actions You plan to do and you're stopping and that kind of state snapshot of your application and testing a complex condition But it's an interesting it's an interesting thing to consider And I think part of the work that needs to be done with with the idea is to build it out more into the large and see what it Look like to incorporate the random core logic test run in your in your test suite And maybe there's a place to include Include things like that to make it explicit what's happening before and after a particular logic run So that's a good question definitely open for further work any other questions great. Thank you


And What is up guys and welcome back to the Johnny Q a daily vlog. Thanks so much for watching today's first piece is a 14 by 14 with these colors Pretty pumped about that and I'm gonna show you what that looks like. Boom.    This is freaking awesome. I with all these colors I'm just gonna try to smear it see if anything happens here Come could look like poop, but we're gonna try it. All right. Let's get this here. We go the final result is oh Baby My goodness look at this piece look at this This is amazing This 14 by 14. Yo, I love it last piece actually got me so excited I'm gonna do it again on a 16 by 20 so good. It's just so good. I gotta do it again. It's just too good Not to do it again. Oh, that was such a good Well, it's not like the pattern but like the color combo like that's what that's what made it really really good Like I said, he's a 16 by 20 and we're trying to remake that last one. It was just so good So, uh, let's get it in this and this it's all right. Let's let's get this All right, here we go. Yeah. Oh my gosh this galaxies through.  Oh, baby Galaxy the Johnny Q like yeah, like oh my gosh, bro. So good man. So good I'm just so pumped right now. So stoked now the 14 by 14 with some retro colors on this piece, so Let's get this That's 14 by 14 with these retro colors. I'm gonna try to do this here Okay, let's get this. Here we go. Yep. Oh, oh, baby Frickin way No, this piece right here Yo Frickin credible. Oh my goodness So good so good since that piece was so good I'll be doing the same colors on a bigger canvas like a 16 by 20 So, uh, let's get this Let's see what happens. Let's get this this 16 by 20 retro colors. Just just exploded This is so 14 by 14.  All right, let's get this the 14 by 14.  I love this so good. So good. Let's see what happens. Here we go. Let's get smooth Look at that. Let's just smooth The 14 by 14 just looks so good So smooth colors bursting flowing. I love it. Love it. There it is


Okay. Hello everyone and welcome to the Mineral Research Lab office hour with me, the host, Justin, and the person who actually want to see Dr. Stringnother. So to start, I would like to have Dr. Stringnother introduce himself before we get started. I think it would be useful to set expectations for what this session is. It's very casual on purpose. We are here such that we can answer your questions, mostly it'd be Mr. Stringnother answering your questions, and yeah, so I will be paying attention to YouTube. I'll be paying attention to Discord just to relay questions over. But otherwise, this is really your time to make it what you would like. So yeah, we're here to answer anything you have. So how about, Serang, can you introduce yourself please? Sure. So I am Dr. Stringnother, a cryptographer and mathematician who is a research contributor to the Mineral Research Lab. Mineral Research Lab is a research and development work group, not the only one, but it's a research and development work group that conducts research and development for the Mineral Project. So that ends up being stuff involving protocol research, some of the math, prototyping, coding, all sorts of things, to kind of push the Mineral protocol and privacy-preserving digital assets forward in general from a technical perspective. And like Justin said, the purpose of this office hour is to give an opportunity to just kind of very informally give a video forum in this case in order to answer any questions that come up, topics that people want to know more about. So this could be Justin and I sitting here in the quiet for an hour, like often happens in kind of standard university in-person office hours, or it can be really whatever sort of technical aspects that folks want to talk about. So I guess I'm using whatever media you I guess have access to. Justin, you said you're watching the Discord on YouTube, is that right? Yeah, so for any questions that people have, go ahead and shoot them off to us there, you can talk about them. Otherwise, let's sit here and have a little coffee. Perfect. I brought my coffee. I haven't had a lot of coffee with my actual coffee chats recently, but I feel like I doubled down on the coffee the last few days. Read the type of person in high school, maybe high school depending, but in college it went to office hours for you the type of person would usually go or is this something you did not go to that often? I did go to office hours when I had questions and later as a TA and even later as an instructor. I don't know. Then I had a new appreciation for the nature of office hours when I was the one running the office hours, but I did discover that oftentimes it was like this really cool combination of folks who came because they really wanted to understand something they didn't. You know, prior to that, but there are also a lot of students who really did know what they were doing and they were just really interested in making sure that they kind of had as much face-to-face time to kind of go over new problems as they could. So one thing I like is that it was like, oh, this is not some kind of sign of weakness to go to office hours. It's like a sign of, I guess, being motivated and dedicated to what you're learning. But there were still many times when I had no one show up to office hours and then it was just reading books for an hour. What are you going to do? Yeah, I guess. Have any questions or any topics of any kind? At the moment, there are no questions and no topics of any kind, but of course it's open. Well, I mean, we can also just talk a little bit of, I guess, something. Yeah, I was thinking one way we could do is sort of troll the answers out of people. We can just start saying like, well, what do you think about this obnoxious thing, just to get people all riled up about it? But before we do that, how about you tell us, talk about what you've been doing with the Mineral Research Lab the last month or so? So I would say probably like the most interesting thing that people might care about, or hopefully at least they'll care about the effects of it, is going to be CLSAG, which is a new linkable ring signature, no, a linkable ring signature construction that was really intended to kind of be a drop-in replacement to the linkable ring construction that the Mineral Protocol used to use. It's called MLSAG. I will say that in hindsight, I really regret us not giving it a cooler name. What would have been a cooler name for that? I have no idea. If anyone has any ideas, you should tell us. I mean, we ended up getting a pretty cool CLSAG logo. Basically, the community came together and we had a few ideas that were pitched, one of which we ultimately went with in the blog post and they're credited at the bottom. But it's not quite as easy to mark it as something like Halo or Arcturus. Because originally, so prior to the confidential transaction model, the linkable ring signature scheme was won by some other authors. It was called LSAG, linkable spontaneous anonymous group signatures. And then from that point on, moving into the confidential transaction model where we replaced in the clear amount, commitments to amounts. Then we moved to one that was developed by Shen Noether and others, kind of more of an in-house kind of thing. That was called MLSAG, is a multi-layered, linkable, spontaneous, anonymous group signatures. The idea there is that you had both information in the signature that dealt with both signing keys but also certain commitment keys. And by cleverly setting up and arranging how you do that signature, you could both show this kind of sign or ambiguous signature model that you're looking for, but also throw in a proof of balance, which is very important, but also completely in sign or ambiguous way. The downsides of the MLSAG signatures of course, you basically kind of have two sets of data floating around in the signature itself. You have some sets of data that deals with like the signing keys and a kind of a separate set of parallel data that deals with the commitment keys. So the scaling on that is not very good. It scales as the anonymity sets per transaction goes up, it scales that way. But at the same time, every time you're adding on new ring members, you're actually adding on two pieces of data effectively, one for signing keys and one for committed keys. And so the kind of the new hotness now, CLSAG, which is concise, linkable, spontaneous, anonymous group signatures. So it is what it is. There was a name that was chosen and thought about changing it, but then we're like, ah, you know, we've already named it, can't really rename it. So people already knew it by that if we didn't want to make everything more confusing. So, but the idea there is to kind of take this information involving these, this data for signing keys and this data for commitment keys. And it turns out you can combine them together in this weighted fashion that involves some hash functions to ensure that someone can't kind of maliciously go through it and run a forgery on the signature. And basically do the same thing that MLSAG signatures do, both show in a signer ambiguous way that you're signing a message on behalf of one of the unknown keys, I guess one of the set keys, but you don't know which one it is, but also basically signing with this other commitment key that you need to prove balance. So it's more or less a drop in replacement, but now effectively you only have to have one set of data involved. So one piece of information per ring member and then some additional auxiliary information that's used just to make the algebra work. So the benefits to this are that it's basically a drop in replacement. So that's great. You know, everything involving key images takes around everything involving the way that these keys are structured gets to stick around. But the benefits, this benefits that you get for it are that first the signatures are much smaller. So effectively, the signatures you get for CLSAG are about half the size as for MLSAG. That's just the signature alone. Transactions include more than just signature. So the fact that they're smaller in that sense, but they're also faster to verify because it turns out you can do some optimization with the way that these operations take place. Because before you had to do some cryptographic operations on like the linkable side of the data and commit inside the data. And now you can effectively do them at the same time. And you can optimize that away a little bit. So benefits there are that you end up with about 20% faster signature verification. So transactions that spend multiple inputs, which most transactions spend between one and two inputs and generate some other outputs. But for every one of those spend inputs, you need a separate signature. And so it turns out that for the most common forms of transactions, like a two input two output transaction, for example, you end up seeing overall about a 25% decrease in the transaction size and overall probably about a 10% speed up in the overall transaction verification. So that's pretty good. There's really no downsides to this. Of course, we want to make sure that the security model for this is very strong. So the security model, it basically just says, what properties do we want this construction to have? And for our particular purposes, we want there to be these properties involving forgeability and non slanderability and linkability. And there's kind of this list of them that you want this linkable ring signature construction to have. So what you do is you basically build this kind of hypothetical model of an imaginary attacker. And you kind of give this attacker different powers to do things. So you might give this imaginary attacker like the power to convince honest users to hand over their private keys, or you might give this imaginary attacker the power to persuade honest users to build arbitrary transactions on the attacker's path. And then you show that even if the attacker had these powers, it still can't go and generally break these properties that we wanted to have without also breaking some computational problem like the discrete logarithm problem that we assume is computationally feasible to do.  Therefore we're okay. So we decided to kind of beef up the security model that was used for CL sag compared to like L sag and ML sag. And in doing so, ended up with I would say like a pretty good model for how we wanted to work. So that's pretty good. We're pretty sure that the same security model would equally apply to ML sag. But, you know, we didn't go back and kind of retroactively do that. It's considered to be pretty straightforward. Do you know if anyone else that is actually using ML sag in a sort of production like Monero is like, is there any other application that's widely used for this? Oh, I mean, besides like projects that, you know, use, use it for the same purpose. Yeah, code forks or, you know, you know, different digital assets that have at least like the same or a similar protocol, presumably use it. But I'm not aware of any other direct applications. So originally L sag, one of the original paper that originally introduced L sag, you know, one option that was listed was like a voting schemes, for example, where you want to be able to ensure that someone can't vote twice for a particular issue, but also some ambiguity among the set of possible voters. I don't think that that's actually been implemented yet. I mean, secure voting is really hard for other reasons. But it happens to be really good for the purposes that we use it for. And so I guess it's got to finish up to the Monero Community commissioned an external audit of both the paper that should be very careful and say preprint. This preprint is still technically a preprint. So that means it hasn't undergone any other peer review aside from what I'm going to talk about. But we decided to have the preprint externally audited as well as the implementation for the upcoming October network upgrade audited. So that was done. There were two auditors who were commissioned to do that. And that was in kind of in consultation with the open source technology improvement fund, which is a nonprofit that does support for these kind of things and supported by Monero Community donations. And the auditors had a lot of really good suggestions for how to improve the overall CL sag security model and preprint. So I ended up changing a few of the proofs around and generally just improving how the security model improves, restructured and run. And those didn't actually require any other changes to the scheme itself. So the construction itself and therefore the code didn't change as a result of that. But we're now much more confident in the way that the security model was arranged, some of the definitions, the cryptographic hardness assumptions and things like that. So that's great. Most changes have been made and that's now up on the IACR eprint archive. And then the implementation surprisingly didn't require any real changes for security, which usually you get some in there. There were a few kind of informational ideas for how to simplify the code. But we considered that changing those, they would have been fairly extensive in how we handle certain key structures and such. And it was thought that it was probably more likely to introduce risk if we made kind of these big sweeping more informational changes than if we were to just leave it. So yeah, so the report's available. There's a blog post up on GetMinera.org about it. You can read the full report, take a look at the pre-print, look at the code if that's your thing. Yeah. Awesome. Yeah, so it's scheduled to be deployed in the October network upgrade. So all you do is just keep your software updated. You use a hardware wallet like the Trezor Ledger. That's in process as well, getting their firmware and apps updated as well to make sure that that's good to go on a day in, day when we're ready to go to. So use those, make sure you keep your firmware updated as well. And it'll be good to go. Very cool. We had a question come in from Andres. He asked if there's any plans to, in the future, move past ring signatures to something that is not just hidden between other decoys, but, you know, has additional protections beyond just a decoy level protection. So I guess, you know, there's quite a few things that I know are involved in this question. One is like, we have the current decoy selection, which per transaction is a relatively small number of decoys per selection, which provides reasonable, you know, mass surveillance protection, but less so good targeted surveillance if someone knows information about particular transactions. So I guess given the current situation, what is the approximate timeline forward? Not necessarily the timeline, but the set of potential improvements forward for whether or not ring signatures are sort of the right approach for the future, and how you've approached dealing with the core problem of the relatively small per transaction decoy levels. Yeah, yeah. So really the question seems to be kind of about per transaction anonymity sets. I increasingly dislike the idea of using the term ring signature to purely mean a limited anonymity set, just because while it has been the case so far that our ring signature construction does have a limited anonymity set, I don't think it's like necessarily the correct term to use. So it's possible to build transaction protocols with limited anonymity sets or full anonymity sets. You could probably do all sorts of other things. I mean, we know there's transaction protocols that I've known and he says, but those are typically not the ones we're interested in. So what we do right now is we do in fact use a linkable ring signature as kind of a building block in a limited anonymity set transaction protocol for the Monero protocol. That's not the only way to do it though. So it's possible to build limited anonymity set transaction protocols that use, for example, specialized zero knowledge proving systems. So I really don't like the whole idea of like zero knowledge, meaningful anonymity and ring signature meaning not because like that's not, that tends to be certain implementations now, but it is not generally true. I mean those things have much more technical meanings involving kind of proof and signature constructions, but using them the way that we do today. So it's possible to kind of migrate over to a still limited anonymity set construction, but one that permits much larger anonymity sets for reasonable transaction sizes of times, but in a way that ideally would help against certain other forms of analysis or attack. Because again, right now the way that the transaction protocol signature scale is that they scale linearly with the size of the anonymity sets. So you're really kind of stuck there. And there's been some proposals for ways to use different kinds of specialized zero knowledge proving systems. Some examples of that are Omni-Ring is one of them, Volantis is another one, Ring CT3, Triptych tourists, there's probably other ones that I'm just not thinking of right now. But all of these essentially allow a transaction proof along with possibly some other auxiliary proofs that scales much better in terms of size and scales a little bit better in terms of time. So verification time is unfortunately kind of the sticking point for this. So if you want a trust-free, like a non-centralized trusted setup style proving system and transaction protocol, you can make those proofs very small. And we know how to do that already. They're not as small, for example, as like the proofs, not the transaction proofs in say Zcash, but they're still quite small for the size of the limited anonymity set you can get. But verification time is always a sticking point. So with those particular kinds of protocols and proofs you do need still like a almost linear verification time. And that'd be the sticking point. So that's kind of the limitation that exists. There are options. They've all got some trade-offs in terms of, you know, what you can do with things like tracing and how the construction works. Some of them involve changes to multi-signature operations, those changes to the way that linking tags work, which kind of require almost sort of like a pool of migration that can still be done safely. And so which one of these, if any, should be the one going forward is kind of up in the air right now. A lot of it depends on what trade-offs people are willing to accept and also what trade-offs people are willing to accept in terms of mainly like transaction verification times.  But that has to be balanced again, what kinds of analysis and attacks you want to be able to protect against. So ideally what we'd love to do is move to something that isn't that full anonymity set. Kind of the classic example that right now is something like, you know, the Zcache protocols, where different anonymity sets that are involved there are basically enforced using proofs that, you know, do things involving Merkel tree proofs. And what that effectively does is it effectively gives you, you know, a full anonymity set within that pool, absent external information. That would be ideal. But right now, all the proposals that are kind of on the table for doing that suffer a lot in terms of either centralized trust or in terms of say proof size or time. Right now you can't really have everything if you don't want to have centralized trust. And of course, you know, there's also other issues with that. You know, for example, in many of the protocols, in many of the protocols, hang on here, someone else joined the room here. Yeah, I think he's probably one of the next speakers that I assume. But anyway, right now, that's kind of the limitation. So under the assumption that, you know, the project in its community are unlikely to move to something that would require centralized trust, you know, right now there has to be a limitation of limited anonymity sets. And so there's still questions about, you know, how do you end up choosing those anonymity sets? There are definitely ways that you can do it that I think provide improvements over the way we do it now. As you get bigger anonymity sets, you can do kind of, you can do certain kinds of binning with the outputs and lasagna anonymity sets. And that can mitigate certain kinds of heuristics involving common ownership. And, you know, the source of where those outputs came in terms of transactions earlier on the chain, as well as timing analysis. So there's a lot of interesting stuff you can play with. But I don't think there's a really good understanding right now of what exactly those precise trade-offs people are willing to make are. Yeah, in my experience, honestly, you know, we have code that can do this now. But I think it's a matter of kind of the community and researchers and developers kind of getting into agreement on what those trade-offs should be. So, and again, hopefully, you know, eventually we get to something that is efficient and trust-free, which would kind of check all the boxes for everyone. From there, you could, you know, very likely build a protocol that uses such a system and does, in fact, give you effectively, you know, full anonymity, which reasonably would be enforced. In my experience, when people talk about full anonymity or limited anonymity, it's sort of in an academic-only way. And then it's applied to, like, you know, typically the way things are currently, right? So limited anonymity looks like a Monero's case, right? And that, in many cases, people think that that just can't even really change. They don't really think about these. Limited is just, you know, anything less than the full pool. And that could be a, you know, a larger number than the entire size of the pool of a different network, theoretically. And then people are just like... Yeah, and it also depends on what exactly the threat model is, right?    And so it's one of those, like, it's one of those infuriating it depends kind of ends where, you know, it's like limited anonymity right now is a trade off.  And we all look forward to the day when we can do something that's trustworthy.  That may come with its other trade offs, you know, based on things like ecosystem availability and, you know, a host of other questions. But it's absolutely a trade off. You know, and I think it's important to acknowledge that, you know, we have things like the break in Monero series where, you know, we kind of try to tease out some of the particular scenarios under which limited anonymity can be a problem. And those for which it might not be as much of a problem. But it's, I feel like, no, it's not fun to like sit down and try to enumerate and think about the specifics of a risk model. But, you know, it's something that unfortunately right now I'm kind of have to do. We haven't had a break in Monero series in a while. So I guess, given that we haven't, what episode ideas would you like to see there be episodes for? Oh, man. I think it'd be interesting to do one that talks about, I guess, some more specifics on things like churn and self-send operations. So I would say things like that, something involving things like output merging would be very, very interesting, you know, where outputs from different transactions end up being pulled into different anonymity sets into the same later transaction. I'm looking at ways to possibly mitigate that. You know, larger anonymity sets with good output selection can mitigate that example. The churn example is a bit tougher because you could look at things like, you know, possible, possible chain history sizes and distributions and look at, you know, what happens with that. It gets, yeah, there's a lot of just interesting things to talk about. But I think it's important to do them in a way that doesn't end up kind of just rapidly devolving into, you know, confusing graphs and, you know, irritating math. But at the very least, I think I'm really, I'm really happy at least to see that there is a lot of interesting research into this area. You know, a lot of folks want general zero-knowledge proving systems that are trust-free and efficient.  And it's interesting to see what different projects do, right? You know, I mean, things like Zcash and related projects get, you know, efficient proofs that are very fast to verify. And you can do things like affected with bull anonymity set transaction protocols in theory. Ideally, Zcash kind of has their whole optionality part to it, but are willing to sacrifice trust to a degree to like a multi-party, multi-party computation, you know, and there's always kind of the question of like, for your personal use case and how you tend to view multi-party computation, you know, do you think that a multi-party computation set up kind of diffuses the trust out enough to where you're okay with that or are you not okay with that? You know, because in the theory that does provide like a guaranteed, you know, this is how the soundness of this operation could fail scenario. If a multi-party computation were, you know, misused, for example. So, so many questions in this space and so many trade-offs. I feel like a lot of cryptography is basically just like the precise study of mathematical trade-offs. But, you know, no, I don't really have a good timeline for when, you know, we might be able to move to something that is, you know, where you consider full anonymity. But there are options for increasing the anonymity set, which lets you do a lot of other cool things, hopefully mitigate some kind of logistics. We can solve everything, but I think it'd be an improvement. So that's on the tip for October, right? Okay. Not for October. So, I mean, there's other interesting things, you know, on the horizon too. Ideas for making range groups a little bit more efficient have come up. There was a preprint that came out on improvement to bulletproofs or bulletproofs plus. The plus actually means fancy. The plus actually means taking away a few proof elements, but bulletproofs minus doesn't sound as good. So, I don't know. There was a proposal to do an implementation of that. It's really new. It's kind of an extension of the way that the bulletproofs inner product protocol works. It's cool. There's a school waiting operation with it.  In theory, they'd be like very, very marginally faster to verify, but, you know, in practice, it would pretty much be a wash. I think in the grand scheme of things. So really the question about whether or not it is worth moving to that is worth the effort and the possible risk of something that's a bit newer has yet to be determined entirely. Were there any other questions that came up besides the one that got us into a very long discussion about protocols? Yeah, there's one, it's still protocol related questions. So it is going to be a follow-up though. It says, so the ideal solution for Monero would be ZK Snarks like thing that doesn't require a trusted setup. And I believe you basically said yes, but also needs to be efficient. Yeah, yeah, yeah. And, you know, and specifically saying like ZK Snarks specifically, you know, it's, there are different ways to, there are different building blocks you can use to construct protocols. And I think it's important to consider at a protocol level. So for example, like you could basically build something akin to, you know, like the Zcash sapling protocol, for example, but without using ZK Snark construction, you could do that with bullet proofs, for example. I don't think it's actually been done, but I know that there were some estimates made to say, okay, you know, bullet proofs is a zero knowledge proofing system. The scaling is, you know, in terms of verification is not as good. Just absolutely not. But what it lets you do is it lets you prove things about circuits in a trust free way. So that is trust free. And so in theory, you could take, you know, the circuit that's using Zcash sapling and some of the other constructions, and you could build that in bullet proofs. And you get rid of trust. And the size would still be pretty good. The estimates and the size would be competitive. Not quite as good, but still pretty competitive. But the verification would be, no, it wouldn't work. Estimates I saw are like still on the order of a second, which were like, well a second, that's pretty fast. It's like, yeah, but then you got to do that like a million times. So you can like amortize that down with batching and stuff. But, you know, it seemed like it was not quite doable at this point. So I would say something that would let you do like the whole Merkel proof kind of thing that, you know, effectively lets you do a full anonymity set based protocol would be ideal if, you know, we could make it mandatory, which other projects have shown you can do that to ensure optimal privacy. But also, I think ideally is making it trust free or, you know, at least abusing the trust and the soundness out to the point where everyone is satisfied with it. And I think it's been the general view of, you know, a lot of folks who support the idea of Monero to just reduce that trust down to nothing.  But that is risky, you know, I mean, like the sprout multi-party computation that started Zcash, for example, did have a soundness problem. And there was a kind of a whole deal where they had to end up taking down the proof transcripts and hoping that no one was able to figure out and abuse that. So, like, there's a real non-trivial risk involved, even to something like such a multi-party computation. So that's not without risk either. And it makes the trust situation a little bit more tricky. So I guess I am personally the opinion that, you know, I like the idea of, I guess, kind of minimizing the possible soundness problems, you know, a soundness problem for something that involves like a trusted setup is the trusted setup and a multi-party computation, you know, for something that, for anything that uses, for example, Peterson commitments, which Monero does, other projects do too, Zcash does, you know, nimble little-based assets do, you know, in theory, if you could break Peterson commitments, that's a problem with, you know, something akin to soundness as well. So it's definitely kind of like this different, like, threat profile. And ideally, we want to minimize that. Got it. Something that was discussed quite a few years ago, but I don't think has really come up super recently is the idea of second-layer networks on Monero. Like, what if you wanted to put the lightning network on Monero? What if you wanted to put this, that, or the other on Monero? What if you wanted to put a, you know, a like ZK Snark mixer on Monero, like as an optional thing? Like, so I guess what, what sort of building blocks need to happen to allow greater compatibility, knowing that right after this, we also have a talk about atomic swaps for Monero too. Yeah. Yeah. Which is a, which is a really cool new thing that's still, that's still in progress. But yeah, so there's some other researchers who were looking at the possibility, what would it take to do, say, you know, atomic swaps between something like Monero and, you know, something like Bitcoin, for example, where in Bitcoin, there's a lot of stuff you can do because Bitcoin has both scripting capability, but also, you know, some different, some different kind of setups in, in, in how it's, you know, its protocol works and in how it runs things like signatures and the like. And that gets really tricky because Monero does not have inherent scripting capability. Adding inherent scripting capability would be pretty awful for fungibility. So I tend to view that as probably a non-starter on the Monero side. So given that the question is, well, okay, so what could you do for something like atomic swaps? And one idea that maybe talked about in the next talk, so we'll try not to give too much away, involved a particular zero knowledge proof that was kind of unspecified at the time in the write up. In theory, we could have done it with something like bullet proofs, but it would have involved hash functions. And that's kind of gets messy to do in, you know, in circuits. So that was not ideal. But then, you know, they came up with another idea that uses this clever cross group proof, where you basically prove that across two different groups, which are, you know, otherwise, basically algebraically incompatible, and you can show a quality of like an unknown discrete log. And it turns out if you can do this, you know, you can very cleverly kind of build it into this protocol involving some Monero transactions and some Bitcoin style transactions in a way that could let me do atomic swaps. Pretty darn interesting. You know, there's still still some kind of open ongoing questions about, you know, how you need to structure those transactions just to make sure that, you know, if one or more parties that are involved in this, and doesn't follow the protocol, you know, what are the risks to funds of anything? What's the worst case scenario that could occur? And even then saying, well, okay, suppose that the protocol does work exactly as you would expect. And the swap does happen. You know, what information that they need is that leak across, you know, one or more of those chains. So, I mean, for example, if you use a centralized service to do swaps right now, like that can obviously have some risk because that entity knows presumably where assets came from and where they're sending them to. So, you know, it kind of is a central storage kind of risk. But also you can do different kinds of timing analysis and other amount analysis across different chains to try to determine, you know, what's going on and what funds are moving where. And if that's a risk for you, then you don't really want that. So, I think the question of what, if anything, about that kind of analysis would transfer from the current setup, which is like, you know, use a centralized service to do my exchanges versus saying, well, I'll just do it atomically without a central service. Like, what kind of analysis could still happen? It's not nothing because metadata always exists. But I think the question of to what extent does it happen? And, you know, is that something that you want to do? I think are still open. Very interesting nonetheless. There were some other ideas for stuff like payment channel networks even within Monero. My DLSag was another signature construction that some other researchers came up with. And we looked at it and unfortunately had like this tracing problem that would have required this whole self spend thing. And it was, it kind of got messy and there wasn't really a great solution for it, unfortunately. So that kind of ended up being dead in the water for that purpose at least, which was really unfortunate. DLSag would have opened the door to some interesting stuff. Got it, got it. I know this is I don't know the LSAG names, just yeah, every letter is involving an LSAG construction just for even more confusing. So I know this isn't your work directly, but I know Istmas recently opened a community crowdfunding system proposal to look into potential limitations in Monero related to quantum computing. He was going to look at the initial scope of what basically challenges Monero needed to address going forward. Can you speak a little bit about what this general work is doing? And then also speak about Monero and quantum computing generally. Yeah, so this is this is work that's ongoing. They believe we're going to work on three months. They're about done with the first month, I think, of that. So again, I'm like not directly working on this, so I don't want to try to speak to them or anything. But their idea was okay, you know, under under the assumption that someday quantum computers would exist. And there's definitely debate on, like, to what extent people think that this would be a problem in the future. If so, how far into the future, you know, and if all of that's, you know, what actions would we want to take now in the protocol to try to mitigate the possible future effects of, you know, quantum computers? And they want to look at kind of different parts of the protocol of range groups and range signatures and kind of a one time addressing construction and all of this stuff. You know, to what extent would different parts of the chain be considered at risk?  And it's pretty well understood that, you know, the one way map right now from private key to public key, where we assume that it's computationally feasible to determine the signing key. Just if you see a key on chain, assume that that's very difficult right now. And it's a one way operation. It's pretty well understood that, you know, given a sufficiently advanced quantum computer, that map would be reversible efficiently. So at that point, like spend anyone's funds, which, I don't know, that's a problem right now. But to some extent, like it's not necessarily just a problem for Monero, you know, this is kind of a broadly applicable problem. The entire Internet runs on these one way maps right now, which will large extent. So it's kind of one of those where your house is on fire, but the rest of the world's on fire too. Doesn't make your fire any less bad, but it doesn't mean that there's a lot of problems to worry about. But even beyond that, looking at things like, what would that allow you to do to ring signatures? So as one example, if since the way that the key images or linking tags work within the Monero protocol, it would allow you to look at different outputs that are part of a ring and determine which of them is the signer, because of the way that the keys work right now. They evolve private keys and you can basically do like a kind of a guess and check testing operation. So, you know, again, under the assumption of a hypothetical quantum computer, this is impossible today as far as we know, you could figure that out. There's some other questions right now involving things like, you know, what could you determine about stealth or one time addressing operations? And that's still, I think, a little bit ambiguous right now. One question that I have that I know is not unique to Monero is saying, well, okay, suppose that I will have a transaction here. Maybe I can't just use that transaction to, for example, figure out what the wallet address of the recipient was. Because remember in Monero, wallet addresses never appear directly on chain, they're used to derive these one time addresses. So right now, I mean, without external information, you can't just use on chain information to link those one time addresses back to the wallet addresses. So the question might be, even with the quantum computer, could you take a transaction and determine its recipient wallet address just kind of with no external information? Or if you, for example, had a candidate list of possible recipients that you think it might be, because, you know, maybe you have a hunch or some other external information, other ways that you could go and basically check those to determine which it would be. And again, the address space in Monero is like a possible wallet address space in Monero is, you know, unfathomably huge. And, you know, quantum computer doesn't just mean like can do everything fast automatically. There's particular algorithms that we know that can be used against things like the discrete log problem and stuff.  But the question of, you know, what could such a computer do if it had like a small known list of possible addresses? So questions like that, I think are still kind of up in the air and are part of the subject of the research that they're working on. And I do know that they're also looking at kind of what current directions my protocol development take that could more efficiently, I guess, work on trying to like mitigate the effects of a future quantum computer. I guess one problem with a lot of algorithms and constructions that are conjectured to be post quantum secure is efficiency. They often tend to be much more inefficient than, I guess, would be reasonable to have on chain. You know, right now, if like post quantum signatures, for example, can get pretty large, and there have been some ideas for how to do some constructions like linkable ring signatures even, but they're very large. And even if you were to integrate them into the protocol, you know, you still deal with today's computer is that, you know, still have to store things and do a lot of processing. And if it gets to be too large of a transaction that's too slow, no one's going to use it. So, you know, ideally, we'd like to migrate the protocol immediately to something that's conjectured to conjectured to be post quantum secure. But there's a lot of parts of the protocol that it's really uncertain if there's anything at this point that's efficient enough. I mean, research in this area is always ongoing. So obviously, what's the state of the art today is almost certainly not going to be even close to what the state of the art is, you know, in 10 years, 20 years, you know, further down the line with maybe we have a better understanding of the likelihood of seeing a practical quantum computer. So I think one thing that we're trying to do and, you know, I don't know, again, I don't want to speak for them, but just my understanding of the work that they're doing is to, you know, look at what are at least some directions in research that might give us an indication about what could be efficient down the road for protocol improvements. I mean, I guess the general thing is like, to some extent, like in the age of, you know, post quantum computers, like a lot of the Internet's kind of screwed in terms of security. So I mean, like Monero would be, you know, at least to some degree, not immune to this, the extent of which different parts of the, I guess, previous chains history could be known. I think the exact nature of that is what they're trying to determine now. I mean, it's really interesting work, but not you think that it's going to be applicable in, you know, your lifetime or the lifetime of people you care about. I think that's also very much kind of a contentious issue right now. I don't think that there's a really universal agreement on, you know, when, if ever folks might end up seeing a practical quantum computer that would affect projects that they use. And I'm not even close to an expert enough in the area to be able to even hypothesize. Yeah, I got it. So, switching gears, what research have you seen outside of what you've specifically done for Monero, I would say, that has been not necessarily useful, but just really interesting and really surprising? I would say I think just like the extensive work that's been done on general zero knowledge-driven systems, I think is really cool. So, let me, right now we use very specific zero knowledge-driven systems in different projects for different purposes. I mean, bulletproof, for example, is a zero knowledge-driven system that does like one particular thing involving Peterson commitments really well. However, like that's just one application. There's a, there's kind of a variant form of bulletproof that you can use to take different algebraic statements, kind of mess them into this particular circuit format and then build a proof over the circuit arrangement in order to prove that you know, I guess, like a witness that ends up satisfying the circuit without revealing what that wins is. And like, that's kind of the general form of like a general zero knowledge-driven system. And so if you have like protocol that you really want to implement, if you can put it into this form and do this kind of representation, then there's always different tools that you can use right now to prove things about statements relating to this language that you built. So bulletproofs can do that. The different proving systems that are used, you know, and stuff like Zcash protocols can do that. There's a lot of stuff involving when CQA starts and, you know, there's a whole host of really interesting research on how to do this and what the different trade-offs are. And like I said before, like kind of the, I would say the holy grail for this is trust-free, small, and efficient to generate and verify proofs. And ideally also stuff involving things like batching that lets you kind of amortize the cost of verification over multiple proof verifications. Like just the fact that there's so much fascinating work going on relating to this is really interesting. And unfortunately, none of it that I've seen I think kind of applies directly to the narrow protocol based on what folks want from it right now. Now, right now the whole trust thing seems to be a pretty big sticking point.  But at the same time, the fact that the research is ongoing and has undergone so much improvement over a pretty short period of time, I mean, speaks really highly to where that area of research is going to go in the future. So I mean, I think it would be great to be able to move to a general idealized future zero knowledge proving system, you know, that lets us build really cool protocols that give us everything we want. Something. It's not today. Hopefully it's not too far off. There has been a lot of work in that area. It's been shocking. Yeah. And to be clear, like, I'm really glad that other projects are doing research in that and, you know, putting that kind of stuff in practice. Like as with all projects, including Monero, like, I think it's important to talk about limitations and tradeoffs. Like, if you tried to do things like this, you're breaking Monero. But it's still good that, you know, those different kinds of applications are also furthering for research. Sorry. I'm also helping some other later speakers come through. Okay. So you have another five minutes. There aren't any other questions that have come in, sadly, just the two from on today's. So thank you, Andreas, for answering those questions. We appreciate them. No one else came to the office hours. They're true office hours. No, that's okay. No, I mean, it's good to kind of just recap, like, how stuff has gone and hypothesizes to where it might go. Research is interesting.  What's going to work. It's not going to work. I think the upgrade in October is going to be exciting because, you know, bullet first was I think a cool example of something where transactions got smaller and faster. And there were really no big trade offs that folks had to consider. And I think CL side is another example where, you know, the transactions get smaller, they go faster, and not to the extent that bullet first did, but and there's not really any security trade offs. You know, if anything, working to CL side helped to kind of improve the way that we understand linkable ring signature security models. So if anything, like, I feel like there's more confidence now about the security of the setup that we're going to have. So anything, it's not really a trade off. It's just, you know, kind of an increasing stack of benefits. You're saying that it would be weird a little bit with whatever comes next, because with ring CT, for example, it was an absolutely necessary change to get to hide the amounts of transactions and all the other benefits came with them. And I think that the issues with denominated Monero were not, they still continue to not be well documented. And I think like a very clear way, we know they're bad, but also, I don't think we've had many research papers yet showing how bad they were. Oh, I mean, the early Monero chain is, you know, a fantastic source of material for analysis. You know, I mean, it's well understood that the effects of, you know, a lot of optionality and protocol has always been problematic. You know, a huge number of early transactions, you know, were effectively, I would say, denomized in the sense of like ring-based signer ambiguity, you know, not again, not based on wallet addresses or anything. But I think that that's kind of still influences work that goes on today. Like there's still parts of the protocol that have some optionality in them. And, you know, you can argue that it's good to have some flexibility in the protocol to allow for alternative use cases that you might not anticipate. But at the same time, like a lot of the work that especially folks like Itzmus and his group have done has shown that the optionality, you know, can lead to fingerprinting if you're not using standard software or using a service that does something in a strange way. And I think there, I think that a lot of folks are coming around to a better appreciation of the fact that limiting the protocol to, you know, improve uniformity and decrease fingerprinting is very much like a one-sided, increasingly lopsided argument toward, you know, limiting the protocol. And there's still things to do in terms of that. And I think the more that we understand about some of the early protocol decisions and how that optionality was bad, I think the more that that can influence better decisions going forward, you know, like the protocol always have like the old chains to deal with. And I think the best to do best thing we can do is try to make the decisions going forward to, you know, improve uniformity that can decrease risk and improve safety. Understood. I'm very sad to ask the last question here. Are there any projects working on voting based in Monero? I know like that one Italian government wrote up that one thing about how they maybe were considered. Yeah, that was that was a political party, right? Yeah, it was taken. Yeah, I don't know if that ever actually got implemented. I don't think it did either.  And if you think you've found out all the ways it could go wrong, like they'll always surprise you. So I don't know. I mean, I thought it was cool to see in the original LSAT paper an application to voting just kind of as a like a fun academic thought experiment.  You know, who knows for voting, but, you know, I don't think that like LSAG or CLSAG would be the thing to use. I mean, there's enough efficiency problems that, you know, trying to scale that out to the, because again, like the whole idea was that you basically have like all the different possible voting entities as like part of a ring or the entirety of the ring. Like that isn't going to scale reasonably well. And, you know, the trust model is so much different for voting that, you know, you could probably get away with something that ends up trading certain kinds of trust for, yeah, I don't know, maybe it's trading that kind of trust for better efficiency. It's, I haven't thought about this nearly enough people will speak well to it. So just keep rambling. Yeah, understood. Okay. Well, that's the time that we have. Thank you so much, Rang Noether. If people want to follow Monero Research Lab, there's always the Monero-research-lab channel. Serang's always there, of course, 24 sevens manning this. Um, yeah, there's definitely the IRC channel and the folks don't want to be on IRC, you know, posting questions in R Monero. There's a lot of folks there who are good at researching development who can answer. On the Monero projects meta repo on the issues there, you can see when all the meetings are and logs from those meetings are always posted as well after the fact. So folks want to see like what people have been talking about. Yeah, and everyone's walking into the meetings if they happen to have anything that they think is interesting or useful research the group might like to see. Yeah, a lot of great contributors, not just me by any means, a lot of really great researchers and folks who are interested in protocol improvements and you know, improving privacy is a digital asset space. So thanks to all the other contributors. Well, thank you so much, Rang Noether, for joining us again.