we're passionate to help you grow, both as a professional nail expert and as a successful entrepreneur. This is why we are delighted to bring you our new high-tech salon innovation so you can step into the future of nail styling. The Pro Nails B system. With many years of experience in the industry, we have developed an extensive in-house research and development lab with rigorous testing of all our products on real-life models. This has led to the revolutionary B system, a 100% vegan and acid-free all-in-one gel system with products for every single nail type. Never before has a salon nail service attracted so many clients, been so time-saving and most of all profitable. The B system, or boost your business system, allows you to perform services twice as fast, thus reducing your service time, allowing you to take on more clients per day. What's more, it provides stunning natural-looking gel nails with our safe, vegan and acid-free gels, allowing you to charge a fair price for your premium quality salon service. This trending nail service is here to stay and is attracting more and more new salon clients by the day. The B gels do the work for you. After a short training, you'll become a B gel system expert in no time. From now on, you'll be able to do nails much faster and without effort, using the B gel drop method in less steps and no filing. The B system saves you a tremendous amount of time and it's up to you to decide how to use that time, more free time for yourself, or fill it with extra clients, doubling your earnings without working extra hours. You heard it right, double your earnings. So do the math on what that would mean in a month or in a year. Let's grow one step at a time and start with only two extra clients a day. At 50 euros each, this will bring you 24,000 euros extra income a year. The B system offers two types of services. The B flex natural nail treatment to enhance the natural beauty of short and flexible natural nails. And the B strong gels for your regular gel clients that allow for longer nails and extensions. Both gels come in various natural tones for all in one semi-transparent natural nude looks. Our 100% vegan acid-free gels ensure a healthier approach to nail care. To ensure you're fully equipped to get the most out of the B system, we offer you a complementary two-day boost your business training with any B system pack purchase. Don't let this opportunity pass you by. To show how fast and how easy a B money is with B flex or B strong, we like to give you a quick demonstration. In this demonstration, we have chosen a B money with the B strong honest. A B money always starts with the right nail preparation and a combined manicure. After all, perfect gel nails require perfectly manicured cuticles and nails. During the boost your business workshop, you will learn how to perform the perfect combined manicure. Next, we apply the primer. Use the acid-free primer for normal dry or damaged nails. Only for greasy nails go for the universal primer. The primer should not be cured in the lamp. Then we move on straight away to apply B flex or B strong. First, apply a base coat of B flex or B strong clear so that we do not damage the natural nail during the next refill. A thin base layer is sufficient. Then cure all five nails at the same time in 30 seconds soft in the smart light. Now it's time to make the B gels work for you. You first apply a slip layer and then a drop layer. As you can see the B gel does all the work for you. The self-leveling gel lays itself perfectly to create the most natural and beautiful nail shape. Work nail by nail and cure the nail each time for 30 seconds soft in the smart light. The B flex and B strong are all in one gels and thus have a beautiful gloss layer incorporated into the product which means that after this step all we have to do is remove the sticky layer and we are completely done. One more pass to sharpen the free edge and make the nails bum resistant. Nourish the cuticles with Vitamina cuticle oil and soften the hands with our anti-age hand cream. Now your client is completely ready to shine. So when do you use B flex and when do you use B strong? Who needs which one? Well let's start with B flex. B flex is a very flexible gel. It's very lightweight and it is designed specifically to bend with the natural nail. You will apply this on clients who are not used to gel nails. Clients who have very flexible but also very short nails and who want to keep this elasticity because it's what they're familiar with. For clients who have a more active profession like they're working in a restaurant they have their hands in the water a lot and they bump their nails a lot they will need a stronger gel or clients who have a longer nail or maybe also want to have a nail extension they will need B strong. So your typical gel client you will use B strong and for people who have never had gel before and who want to keep this natural feeling you can use B flex on the condition that the nails are short. Then we have the category in the middle the Sopores clients the semi-permanent clients. These are also clients who are used to the feeling of their natural nails so these clients would also be suitable clients for B flex. And there's more. B color completes the B system with its highly pigmented one-layer color gel thus saving even more time. B color is the salon color gel of the future. Its one-layer application is highly pigmented ensuring full coverage with minimal effort and maximum time saving. Designed for efficiency B color applies smoothly without streaks or clouds and it does not retract. It allows for simultaneous application and curing of all five fingers. It's the ultimate time saver curing in 30 seconds full in the smart light leaving no sticky layer. You finish it with a layer of B flex clear or B strong clear curing in 30 seconds soft for the glorious B system glossy shine. Try the B system for its unmatched efficiency beauty and business growth potential with our comprehensive training and innovative products you're set to redefine your salon services and let your business thrive. Let us also show you what a delight it is to work with B color. If your client prefers color instead of a nude look you can finish your B Manny with the B color of her choice. This one-layer color gel is so fast that it only takes an extra five minutes so you can apply a color in half the time. After removing the sticky layer of your B flex or B strong gel continue with applying the B color. This is done in one layer which immediately covers perfectly and is very easy to apply. The color does not retract so you can work five fingers at once and then cure all fingers simultaneously in 30 seconds full in the smart light. B color leaves no sticky layer after curing. You will finish with a layer of B flex or B strong clear for that beautiful B system shine by curing it in 30 seconds soft in the smart light. Here too you can apply it to five fingers at a time and cure the entire hand in 30 seconds soft in the smart light. Finish by removing the sticky layer and that's it. Check out the final result. With B color you can give your B Manny a beautiful color in no time thanks to the super fast and hyper pigmented formula.


Hello, I'm Hayato Saigo, okay. Just wanted to make sure it's going, it's all good. All right, welcome to both. It is June 13th at zero UTC. We're here in active guest stream number 44.1 with Hayato Saigo and Nausugu Suchia. So really looking forward to this presentation and discussion. Hayato, please say hello. And then we'll continue with Nausugu's presentation. Hello, I'm Hayato Saigo. I'm a mathematician and a mathematical physicist also interested in consciousness. I have worked a lot with Nao by using category theory and application to IIT or other consciousness theories. So Nao. All right. So my name is Nao Suchia. I'm a professor in Monash University and also having a lab in ATR in Japan. And as a sort of the introduction to this, to the session, I'll give a brief talk on our recent project called the Korea Structure, which Hayato and I will be collaborating and also a little bit on the quantum-quadra hypothesis. Both of them are hopefully relevant to active inference issue. So the Korea Structure project is the Japanese funding that we received recently. And it's going to continue for five years. This will be to establish a super interdisciplinary research program to tackle the program of consciousness, especially Korea through the interdisciplinary research. And we are planning to also host summer school for two to three weeks every year. And here's the fantastic members for this grant. And Hayato is basically working with me in the part of the mathematical approach to the consciousness and the Korea. And so one of the problems, for example, we want to address is, for example, when you see this square, Daniel, what do you see here? Blue. Yeah. So you say blue and I say maybe owl in Japanese and Hayato may say suni in Russian. But we could mean the same experience, Gloria. But how do we know that we are actually experiencing the same thing? And I'm actually curious whether this kind of question is something that the free energy or active inference kind of style approach can ever tackle. And my approach is to go into the category theory kind of inspired direction. And here's the first study that we published a few years ago with Hayato and Shigeru together with also another category theorist, Steve Phillips, that in the category theory, there is an interesting lemma or the Yonezarema that proves that all the relationship characterize its internal intrinsic property. And we are using that idea to study the Korea. That's the kind of inspiration of our approach. So instead of trying to characterize what the blue is or red is, we are trying to characterize all the relationship between a particular color or a particular quality to other exhaustive. And then by finding the correspondence of this relationship, we try to identify what the actual blue is. So if you would say blue is similar to green but also similar to red, then that's going to be very different from my blue. And that's the way how we detect this blueness. It's same or different. And one recent application of this kind of idea is this unsupervised alignment that we are using with another collaborator, Masahumi, always me, in University of Tokyo. And I'll skip the detail, but basically what we are doing is to do some kind of experiment and then collect data to obtain so-called similarity matrix between all the colors. And then without really knowing what color maps onto what, we try to use only relationship to predict what we are seeing. So in a sense, this is a purely category theoretical kind of approach to the problem of consciousness. Then I'll just skip the details here. And then, yeah, so we are doing this kind of approach on the other types of the Korea. And then this is the thing that we are planning to do for the next five years. And then another just brief kind of introduction. If you have some other question, we can come back later. So this is potentially more to do with that active inference aspect of our approach in a sense. This is called the quantum Korea hypothesis. We published this preprint really recently and so you can access it later. So there are many programs in the current models of our Korea or our consciousness where we tend to think that experience or consciousness is like one point in the high dimensional space. And we use that kind of idea for our previous alignment experiment approach. But it lacks dynamics and it also lacks the effects that we tend to experience where the observation or report or introspection itself sometimes changes our Korea. And that's the sort of the active inference aspect that I'm talking about. Attention or introspection changes our experience. But the traditional approach is very difficult to deal with this. And one way to deal with that is this quantum formalism that we think is appropriate. So here we think that there is some kind of Korea experiencing system that is defined as this blue boundary. You can consider that as sort of the Markov blanket like thing in the free energy kind of idea. And then we consider a particular type of the quail like redness or blueness itself as an observable. It's a terminology in quantum theory. Something that in principle can be observed within this Korea. And then we consider this measurement as a process that probes this Korea's observable. And unlike classical theoretical thinking, we think that the measurement or probing of this observable through it's called state changes its states and also observable itself. And that's something that we can actually formalize mathematically using a concept called a very general concept of this conditional probability called instruments. It's very technical for the quick talk. So I'll leave it, but this is a modern quantum theoretical idea that captures both how we can affect on the world and that quality itself can be affected by that measurement. And then using this, we think that we can explain probably Korea in a much more reasonable way with the mathematics and mathematical formulas. And so far it's just a kind of theory, but we already think that it's probably better than the existing kind of a model of the, for example, attention and the consciousness and also it also provides some kind of prediction and explanation of why some of the two concepts or two experiences can feel different depending on the order that you experience. So this is just a conceptual kind of example. But if I ask you, Daniel, that is China similar to North Korea, then you might say probably no, right? I don't even know the reference point. Yeah. Okay. But let's say, you know, just subjectively, you know, China doesn't feel similar to North Korea. Okay. Yeah. However, if I reverse the order, is North Korea similar to China? Then many people actually start to say that, ah, yeah, it's kind of similar. Hmm. Does that have to do with one being larger or what is it about the order that primes people to respond differently? Yeah. So many people have already, you know, came up with many different kind of ideas to do, explain this kind of individual aspects of this, you know, order effects. But as we mentioned, you know, we are actually computing or, you know, measuring this kind of order effects systematically through all combination of the colors, for example. Then it's hard to systematically explain why this order effect emerges in one case, but not the other, and in which direction, right? But that's what we found basically. And but using this, you know, quantum combination kind of framework, we can explain it really nicely. Hmm. In this case, it's just a projection substrate, but we are probably going to extend much more in the more instrument like, you know, a framework so in the future. Yeah. So, and also we can, we are also predicting that there may be some kind of a violation like a violation of inequality like situation in, you know, perception. And if we can confirm that, that also, you know, gives a credence to our proposal that the quality is like, you know, needs to be framed or explained in the quantum framework. So I think that's probably enough to start up the conversation, I guess. Awesome. Yeah. Okay. Great. A lot to jump into. I'll share my screen so we can take some notes. Okay.  Or I'm happy to, but do you have a first remark or what parts of the work do you want to reflect on first? I think maybe the categorical approach, the importance of the categorical approach is one issue to talk with you, I think, because you first pointed out that paper with Shigeru Tabuchi in 2060. So I'm interested in why you and your colleague are interested in that paper from the active inference side. So can you explain some feeling? Yes. So this is using category theory to assess the relationship between consciousness and integrated information theory with the two of you and Tabuchi from 2016. So I've long been interested in scale free or scale independent ways to describe systems and information theory has been recognized as one of those approaches. And information theory has been proposed to not just play, of course, a central role in the functionality of information systems, but with IIT and related theorems to also strike at the heart of the most quintessential aspect of our conscious experience. So that seems like an incredible synthesis of the functionality of systems that don't necessarily seem to have anything qualia like at all merely performing calculations like a super fast abacus basically. And so I am curious and feel free to address either of these things. When we describe information systems, what is it that would or could or is experienced? Are they experiencing their information what they are processing or what is it about information integration would grant a system the ability to move beyond a behavioral functionality into again strike at the heart of one of the most ineffable questions that the mind can really consider. Okay, interesting. Actually the category itself in mathematics originated or created in mid-20th century to breach the two different fields that is geometry and algebra. More specifically the topology you know something about topology maybe which is a very general kind of mathematical methods to capture the essence of the figures like a circle or the complicated figures and the fundamental problem of the topology is can you identify the two different figures essentially that means you can you transform continuously to the other figures or not. So if the one figure goes to another figure with a continuous transformation and back then it is viewed as essentially the same. But to determine the two figures is the same is a very difficult question. So like great mathematicians figured out the utility of the algebraic method that like making the one figure to the other from the figure you can construct the algebraic system like groups or something and then the transform the problem into the algebraic region. Then the point is very essential point is taking one figure to the constructing one algebra from the one figure is not sufficient. The point is the relation should be transformed the other regions. So one region here other regions here but one to one correspondence of objects is not sufficiently but the relations if you can transform the old relations into the other regions then you can provide you can prove something from one region to another region because yeah this is a notion of the factor like the region intuitively I talked about the term using term the region it is mathematically called category. Category consists of objects and arrows or morphism which is a generalized kind of relationship between these objects. So topological region is some kind of category of topology like a top that objects are topological space and the continuous functions continuous mapping is arrow or something the other regions for example algebraic region like groups and group homomorphism or the module vector space or something so something algebraic and the relationship between them.  So first the Arlenmagen-McBren define category to define this notion of functor by the bridging to region then if this is established then you can infer about topological region category by using the algebraic the relational something. So we that is now Shigeru and me talked about the possible application of this kind of argument into the consciousness study that it is just a problem of bridging different kind of regions like a physically testable region and our role experience itself or something so it is a apolia it seems apolia it is a too difficult question too hard problem to connect to these regions but if you only focus on the object but the relationship if you can compare the relationship then you can infer something about the you cannot directly touch you directly manipulate. So this is the idea maybe I'll just add a little bit so yeah as Hayato said under you you are taking really diligent notes so that may be quite good but from my end so this paper was inspired by really talking to Hayato and so Shigeru and meeting in this summer school on consciousness and there I encountered in this category theory for the first time and there were several shocking kind of you know concepts that I learned from this first encounter one is that there isn't a way to compare two completely different things and then extract the common structure between them and Hayato first introduced me this you know very strong strange paper which I didn't understand at all but it was so interesting that I could read for hours and hours it's called the Rosetta Stone it points out the commonality between topology, quantum physics logic and computation yeah that was the title and then I didn't understand what the heck is this paper but that uses this category theory and as a whole I feel like I see so maybe if we understand the structure behind each of this concrete example we may be able to find the commonality or explanation at that level and that naturally applied to my program at the time which is how to bridge the gap between the quaria and the information structure derived from neural activity this looks also completely different but if there is any possibility to bridge the gap it must be some kind of structural mathematics and that's this category theory and one point is another point is that you know I think you can get the ideas and functions by bridging two regions then the really interesting thing another really interesting concept is called natural transformation natural transformation is the kind of deformation or transformation between transformation so you imagine some theory bridges some region to the another region so there is a one meta-level R code but another theory predicts another thing or something but can you connect these theories or models this is a program of natural transformation so one representation one theory or something may map the one region to another region but there is a there may be another various kind of mappings, functions so the problem is to compare or to unite to connect this transformation is a function is called natural transformation if the function is naturally isomorphic that means there is a reversible natural transformation between functions then these theories seems different but essentially the same so kind of natural transformation is maybe the hint to connect the theories on consciousness or something and also the our consciousness experience itself will be connected to the concept of natural transformation because I see something the different thing something but there is no divided discrete kind of structure but there is continuous or relating something so this view is something like another view is something like but there is no division there is some relationship so our experience maybe itself a kind of product of natural transformation like maybe now you can explain like a maybe now you are muted is it alright if I draw some kind of figure of course would you like to share your screen and also if you can give me the access to the annotation okay I will make it so everyone can edit and I will put it into the chat here and then also if you want to draw that would be awesome looks good so just to kind of recap what Hayat was talking about is that we tend to think about something like a category and then another category is there and then there is some kind of object which could be quite abstract and then it's related by arrow like this and then if this category C is somehow possible to map onto this category D while kind of preserving this arrow for example this arrow goes to here and then this arrow goes to here and then each object is also you know preserved then it's much more than the usual kind of idea of a function between the sets right because it also imposes the correspondence between the relationship and in our idea basically if this is let's say the quality of the category and if this is a neuronal activity or maybe information domain and then maybe you know we can have back and forth of the function that would be great usually the most of the mathematical theory of consciousness tries to go from here to here right let's say in the case of IIT arrow here and then what Hayat was kind of talking in a way as a natural transformation is this kind of you know secondary level arrow let's say I don't know whether free energy or active inference can count as a prediction from you know neural activity to that particular quality but let's say if there is something like that and then maybe there is some kind of a translation between these two theories if that exists that's called a natural transformation awesome well a few comments on this the idea of mapping the maps between the spaces is so important it reminds me of metaphor where we're conveying relationships and saying well it's like a hammer or is a hammer for this other situation how we get few shot learning and applicability and how we grasp concepts is perhaps developmentally or from an evolutionary perspective in a large extent by generalizing our spatial and embodied understandings and then certainly communication if you want to have some semantics that are being communicated across minds through an interface through a classical interface then certainly you want the structure to be preserved and then I think the one that you have here is one of the most provocative for those empirically studying consciousness because it's kind of like this side is empirical and let's just say we don't know what the Q side is for now but let's just say it's another space that isn't that and then if there is a certain landscape that neural activity patterns flow through like some kind of winterless competition or you get some sort of limit cycle and you find that there's a limit cycle of neural activity and then there's also that is being paralleled by experiential cycles then is not just an accounting system that's a very strong finding about potentially the real underlying causal system especially because it can suggest unique other explanations or manipulations to make Do you want to mention anything about the metaphor? Before that very hard to say the technical remark is that for the original problem that connects topology and algebra in that case the algebra is more concrete more manipulatable in that case and topology is something mysterious in this case so in general the function is a kind of generalization of mapping in sets so there's something mysterious into something concrete so now picture C and D should be switched maybe in that case but of course there is a back and forth so the idea is your blue and my blue program is for example you can prove that your blue and my blue is different if there's something informational or something detectable structure is completely different then your blue and my blue is different of course you cannot prove the sameness between your blue and my blue if there is no how to say back and forth function but even if you don't have the back course function at least you can demonstrate that your blue and my blue is different seems to be different if the correspondence is functorial or something so it is a big step from the experience side and with me and colleagues do such a thing so your blue but this is not by focusing your blue my blue only but your blue read something or you don't have to say even do not have to say the name of something so just no labeling for example but you can teach the relationship these things then compare that so very interestingly in general the structure is surprisingly coherent between persons so it is very nice and also maybe it is suggesting that the non-isomorphic case then your blue and my blue is something different so I think this approach is very interesting even if there is no bi-direction only one direction then even that case it is useful it is my remark please this is one of the slides that I didn't show today what Hayat was talking about is kind of explaining this figure the way you experience red is related to the other experience the color but shape and sound in a very complicated way and then we can measure how similar they are by asking the people and then you can also do that for the other people and then these two similarity can be directly compared with our new method of non-isomorphic learning and IIT for example may allow us in the future to go back and forth between this quotient structure and the information structure and then mathematically we believe that the information structure should be potentially translatable in some way and if that is possible in some way eventually we may be able to say the way you are experiencing red is same or different this way or that way that's a sort of idea can you go back to your drawing awesome so for the top arrow just exploring what might it mean that IIT is going back and forth and also that active inference is going back and forth what does that mean C and D represent in IIT is that a quantitative evaluation like the five variable or what is being accounted for with that arrow I have to do you want to say something there is various possibilities for example the sameness or difference so if you have some kind of gradation of sameness so yeah sameness gradation but to formulate the gradation gradation of sameness you have to move on the enriched category not a simple kind of category but it's generalized kind of category called enriched category in that case for example the simplest example enriched category is generalized generalized matrix space matrix but in matrix space for example the point P and P itself has a distance zero of course and kind of P to Q R is there is a triangular inequality or something so yeah then it can be considered as a generalized kind of compositional structure called enriched category it is a typical kind of enriched category so but symmetry does not always hold or something so this is one how to say one small step to grasp our core structure because like now pointed out North Korea, China distance and China North Korea distance is in general different or something so almost all studies assume that some symmetry we need to go beyond that and then maybe the one number or something is too poor to represent everything so you have to go more and more richer structure like not just numbers but some kind of projection operator or something then you enter into the quantum region so relationships we feel maybe should be represented from some kind of relationship between the notion of state in quantum theory and state transition between that so our approach is step by step and we use the many arrows to study something and then goes to unify the more and more general kind of arrows and categories and this is just for the speculation but this kind of quantum theoretical like as a mathematical framework do not need to but the kind of mathematics can you use unify these kind of things awesome just a few comments on that when we're doing free energy minimization we use the KL divergence which is very importantly not a distance because a distance would have the symmetry characteristic and so the KL is chosen because it's easy to fit optimization on these variational distributions but it's almost like it's a feature not a bug that the KL divergence between Q and P is not the same as P and Q because it turns out that that kind of a divergence based quantification actually has some advantages over distance measure and then the mapping between observables and hidden states and the cognitive operators that we bring to bear on that kind of mapping problem those are some of the most fundamental pieces of the partially observable Markov decision processes that are used in the active inference generative model which is first the mapping between observables and hidden states and then how do hidden states change their time and how does action selection policy selection influence how hidden states unfold through time so this mapping between hidden states and observables and between hidden states and the hidden state at the next time point are and how again how action influences that if you understand those operations then it's like a game of chess and the ball just rolls downhill and the variational distribution works exceedingly well but when those are off base then you're fitting your way like right off a cliff so maybe then I'll just add one more point so you asked in the original like 5 minutes ago what each of these points or arrow correspond to in the case of the IIT and so on and then this is kind of related to all these ideas but one of the fundamental insights from IIT but also from philosophy especially some kind of philosophy called process theory it is important to really understand that the process or flow or movement is really fundamentally important kind of aspect of things and then that may be really distinguishing some kind of static sets or the thing in the static world when they start to interact and then when the process emerge then something starts to potentially generate or support subjective quality like proto-quadra kind of thing in the case of the IIT it is deeply entrenched in the formation of the transition probability matrix of an entity like how this thing neuron or logic gate it might have come from the previous state and it may go into the next state so any kind of a moment is already embedding the possible previous and the possible future there is some kind of a width in the moment because it's a process and when the process is considered then as Hayato said that this kind of dot is a thing kind of structure to think about so we need a more elaborated structure of a dot itself but the category theory is great in the sense that each dot can itself contain complicated structure it's just you know represented as a single dot but it can contain something more so in the upshot what could correspond to a collection of the neuronal states and the connectivity pattern and things like that which can go from one to the other and then this kind of movement may correspond to something like we see redness or whatever in this way but that's also with respect to other kind of arrows as well the wave of the relationship that we are trying to characterize both in the domain of the aquaria but also brain is also the wave of the relationship gigantic ones right and as you said that you know transition from one state to the another state that's pretty much like dynamical state dynamical system like you know approach and then the simple idea called is applicable to this kind of dynamical system approach correct me Hayato if I'm saying something out of the bounds of my comfort zone but any kind of state can transition into another state and then this transition itself can be represented by the arrow and yeah so kind of the combination of that particular kind of arrow composition may correspond to one experience as well so that's one thing I wanted to say the other thing that Hayato also wanted to mention about observable states and why we are interested in the quantum you know framework quantum theory by the way it's not a quantum brain hypothesis we are not saying that you know quantum phenomena in the brain is critical for consciousness but it's just a mathematical structure of the quantum theory that is important to understand the qualia and the essence is most likely to be this you know non-commutativity any kind of process that you do like you know cooking you know putting the water and then model is different you know model and then putting water right and so any kind of process we do there is some kind of temporal dependency that commutativity doesn't hold and that's also probably the case for the qualia and if that's the case then for you maybe it's a bit surprising but you know Bayesian kind of statistics and Bayesian probability may not work as you might wish yes so in other words using the quantum mathematics there is a quite good generalization of Bayesian innovation or Bayesian kind of inference yes actually the one step generalization is called projection in kind of mathematics there is more generalization called instrument so instrument is general notion of measurements like a system and environments interacting then more precisely the measurement is something like the interaction feature has some probe of the apparatus so probe is some kind of the apparatus is some very small tips or something which can be considered as a quantum, this quantum system and this probe system interacting quantumly then we can measure this probe by the macro side of the apparatus so this is a very general kind of measurement which is called which is identified mathematically with the notion of an instrument originated by the Davis and Louis and Ozawa one of our collaborators we begin in the cooperation but Ozawa who is also famous for the Ozawa's inequality Ozawa's inequality as a generalization of the Heisenberg universal value version of Heisenberg's inequality but he identified this general notion measurement and instrument called completely positive instrument in mathematics but this CP, completely positive instrument is quite general and generalizing very beautifully the Bayesian influence into the quantum side so I think the using, focusing on this kind of instrument kind of concept is the key to bridge the RTP per side and our side and quantum side or something so it is my speculation but I think it is a very fruitful direction It's awesome, just one comment on that the probe with the instrument and the influence frame I believe as Chris Fields might describe it is an experimenter's choice and so the way that we could talk about policy selection and inference on which experiment to run would be in terms of which sequence of observations is going to minimize expected free energy so the sequence of observations is going to have high pragmatic value and epistemic value that's the expected free energy functional now as researchers we might seek out the most informative experiments and not prefer to see a certain measurement so we can say well all measurements are a priori equally pragmatically valuable to us so now expected free energy is going to be driven by maximization just by pure learning but this recognition that action is woven into the process of reducing uncertainty about other systems across blankets it's absolutely fundamental because if you don't have the constraint that you need to actively select which experiments to run and to probe and to interfere in this quantum way, this way described by quantum mathematics you're kind of in a delusional alternative world where like you get information from photons but you don't have a sensor so it's not an empirically tractable world unless we take on this constraint and then once we do our mathematics are the quantum mathematics and again it's kind of a subtle point this isn't the orco or this isn't necessarily taking a stance one way or the other on what kinds of electronic scale quantum phenomena exist in any system but rather just sees quantum mathematics as fundamental to decision making and uncertainty which is quite a change from how it's been discussed for a hundred years actually as Nao said we're not dependent on the hypothesis like there is a quantum physical process behind the Korea but why we can use the quantum mathematics is very very simple reason because conventional probability theory classical probability theory depend on the probability space classical one is too narrow to capture for example all the effect or something but you can generalize it so kind of mathematics is just generalize the classical mathematics and I think I believe that why quantum mathematics is found for kind of physics is very simple reason because in classical physics some kind of determinism classical determinism is very universal and the probability only comes from some kind of ignorance of us but in quantum physical context there is some kind of very fundamental randomness fundamental accidents or something is found and they are all surprised because if you go into the micro side the determinism should be stronger they thought but actually there needs some kind of fundamental non-committativity in the micro side but without a bore and Heisenberg or some people pointed out that even in classical or even in a macroscopic side but this kind of the importance of non-committativity is very universal not especially in micro side but it is a historical reason that we first find in quantum mathematics in the micro side but there is many many phenomena which cannot be captured by the determinism or the classical probability theory so I think the courier or something is just some kind of example by the way maybe Daniel can you replace this quantum consciousness with the quantum courier the quantum consciousness has been used to imply this quantum brain hypothesis which may be potentially true but so far we don't really see the strong evidence about this particular thing what's funny I'm thinking of the classic qualia question what is it like to be a bat and it's like what if the like is interpreted as a metaphor what if there's a metaphorical mapping to bat which is an observer dependent mapping amongst spaces because we've moved the focus from the objective standalone experience of the bat to considering mappings amongst cognitive systems one critical one being for example whoever is listening to the question and a bat who's the one who actually answers that question and you asked also about the inspiration from the category theory relevant to the issue of the metaphor of communication so now it's a bit far from this quantum courier and jump back to another topic so the metaphor is something that Hayato has been working on so but the other part which is communication and that's something that we also wrote together last year and using this concept called adjunction in the category theory which may be interesting to discuss so in 2016 paper we already mentioned that category theory is very promising for the study of consciousness precisely because it provides different levels of sameness when I heard this from Hayato I was like what do you mean but simultaneously it makes a lot of sense and in fact many controversy or confusion among the field of consciousness but not only for consciousness, other fields like ecology as well classification of the animal species and things like that all depends on this what do you mean by the thing and this is something that also relates to what you were talking about a moment ago which is observer dependence so even at the very bottom of this reality what matters is some notion of this subjective choice of what it means to be the same equivalence or adjunction is a particularly interesting kind of situation where for example integer and real number is definitely different but in terms of the relationship if you start to think about the relationship there is some kind of one-to-one correspondence between them and also Hayato's book also written in Japanese but recently I read it and one of the interesting examples is also multiplication and exponentiation at the level of object or number it doesn't match but there is a very nice relationship between them that is very fundamental and something like this apparently completely different thing but finding really meaningful relationship that is very fundamental in communication we are talking different kind of language with different kind of meaning but because when I say something you reply and there is some kind of systematic difference between what I expect and what is the kind of prediction error I think the prediction error may not need to be minimized but as long as it is consistent and systematic then it actually works as a very nice translational kind of device after living like 20 years more than 20 years of living outside of Japan very strong English accent either in Japanese or living in Australia or in the U.S. I can't minimize this prediction error but because I have this systematic prediction error for you you totally understand me something like this is the interesting thing about that junction this makes me think about nested reference frames in communication and let's just say that the expected free energy or maybe even the integrated information content is such and such a level but it varies within some range it is 100 plus or minus 10 instead of just trying to naively minimize that to zero we can create a nested model such that we expect that to be 100 plus or minus 10 at which point the non-zero value is stabilized or buffered and so then that becomes the kind of stabilizes or wraps the fact that we're not perfectly transmitting information but we can be perfectly aligned that we're not perfectly transmitting information and also our language understanding for fluent speakers we're able to map the phonemes a little bit deeper we're able to read between the lines or listen between the phonemes so that people who say words differently map to the exact same syntax let alone the semantics the sound gets kind of abstracted out but it influences our qualia so something is being passed forward with like tonality and the timing and I wonder about with of course large language models how much context they're able to garner from the string of tokens when for our experience sending and receiving natural language communication it's all about the timing and the tone and the setting and so finding ways that we can describe that type of information the category theory capacity for the different levels of sameness seems like exactly the tool that we need I have to do you want to add anything? yeah I'm thinking about about that you know the multi-level kind of free energy I actually have a question to you actually so I recently discussed with a roboticist in Japan Jun Tani you know his work is using this predictive coding kind of scheme for constructing the self-autonomous robot and he thinks this intermingo of the top down prediction and the bottom up prediction error is the place where you can see a lot of Korea emerges and things like that but one of the most interesting aspect of his model to me is that he has this different time scale for his neural system the most peripheral motor level and sensory level has a very fast time scale so that tries to match with what they want to do towards you know outside and the intermediate level has a slightly longer time scale so there is some kind of inertia and even higher level has a much much longer time scale and so in real probably neural system there is also different level of the time scale and also different levels of the mechanism acting at the same time changing the activity state in spiking and also release of the synapses, synaptic transmitters that could increase or decrease over the time and also even the new neurons emerging and going out so the minimization when you are talking about minimization probably that kind of minimization is actually happening independently and also coherently potentially or contradictory across different levels and also regions and there may be some kind of effort or whatever to also coordinate these kind of contradictory demands of the minimization and what do you think about this issue?  That's a great question well one thing that stood out to me was you described the slower moving levels as having inertia and from the Bayesian mechanics perspective that's exactly how we would model it they're slower moving, they have more inertia and they move more like classical objects whereas the faster time scales are moving more like lighter particles more like that quantum probe in the limit and then this sort of center or the highest nested model would be like the most cognitively massive potentially the slowest potentially integrating information over the longest time window and certainly this is underlaying by these multiple time scales of mechanism spiking neurons from a few hundred milliseconds time scale, neuromodulatory processes over seconds and minutes synaptic plasticity even neurogenesis and the development of our brain. So what does it really mean that minimization is occurring when we have this heterarchy where we have similar types of free energy minimizing units or active inference units within a level like cortical columns or grid cell units and then also we have these nested models that are optimizing across spatially separated scales. Well one paper that looked at this is our live stream 42 on the slam simultaneous localization and mapping also in robotics and so they had a two nested model with two steps and the lower level was like the tactics and that was where you had the motor and the sensors and then the nested model was slower that was like where do I want to be that was more like the strategy and you can create an expected free energy functional that sums across those two levels. How you tune the parameters is going to matter because how do you wait? Yeah yeah like and one what this comes up with like preference learning like if you have if your preference if you have to left or turn and right turn and your preference vector is like two and one versus two million and one million then that is the ratio of the preference is the same but the amplitude of the preferences is a lot different which is going to swing the model towards valuing pragmatic outcomes whereas if you had 0.02 and 0.01 the ratio of the preference would still be the same but it's such a small the amplitude preference and so similarly with expected free energy minimization you could make it so that the joint minimization was driven by just the top level of the model and then the sort of contradictions or even increases in expected free energy of the lower sensory model would basically always be accepted in the service of the top model or vice versa and it seems like biological systems with their sort of precarious or self organized criticality operations they're poised where you can have like propagations where small changes in the bottom up model can propagate and induce like phase changes of higher levels and vice versa so they're certainly poised parametrically at a special place where the composition of the faster sensory motor models and the slower like cognitive or narrative or metacognitive levels it's all poised in a very integrated way potentially even one that is where the IIT measures tell us something useful but yes the contradictory demands of subunits within a level of collective behavior and across levels as a biologist I think we just point to ecology, evolution and development just distributed systems that don't exist at an adaptive point or implement adaptive dynamics amidst uncertainty they just fail to exist and so the complex biological systems that we observe today do manage that trade off and then in robotics they have this unique challenge of developing systems with one shot or with iterated development that do have some of those properties but for sure there's no reason to think that just by laying out a nested active inference model that you would actually get like a more adaptive decision making agent in fact as you nest models the state space of these models becomes vast and with some back of the envelope calculations you can see like well if I do this level of nesting and have this kind of sensor array we need this absurd amount of data to actually parameterize this model and by the time we collected that many observations maybe the causal structure that we learn to the world 200 hours of video go don't apply yeah I mean that could be quite a huge difference between probably IIT and free energy I think the IIT explicitly states that this time scale that matters but in this case it's just about in Korea is a particular time scale that maximizes integrated information and that corresponds to the time scale of our experience like TANI's model that slowest time scale on the order of 100 seconds or something like that and also you know synaptic plasticity is on the order of hours and most likely spiking neurons seems to be more mattering with respect to our particular experience at each moment right and why it is the case that the provisional answer provided by IIT is that because at that particular time scale integrated information is maximized and across the scale and across the regions but this is a huge conjecture and nobody knows whether this is true or not and when we use the fly model we weren't able to confirm any kind of clear peak in the integrated information using a different time scale so it's still something that I'm very interested in looking but still not clear whether this is the case I don't know if Hayato has some idea or insight into the appropriate time scale for consciousness or something from the physics point of view I have some thoughts but maybe it becomes a long story and also connects to the observa dependence program so I was thinking about how to explain these very important things in one thread and I found that first the very simple fact that there is no there comes no identically same two events every event is different in some respect but to do science or more generally some recognition of law we need some criterion of sameness for example you want to know how to heal the disease you should identify the different persons people as the same kind of samples of course there is no identical person but you have to identify the same in the members of the same group so all the scientific studies depend on some choice of sameness in that sense all the science depends on this kind of choice of sameness as you both pointed out and there is no I think there is no canonical choice of the sameness you can choose every sameness but if you choose some sameness then you can talk about the observables observables mean some kind of quantities you can observe or something maybe it randomly changes the value or no value like a kind of case but by measuring process it becomes half the value or something so there always may be indefinite nature of such kind of quantities but anyway we can imagine that quantities has some kind of structure between the quantities so in mathematical terms it will be some algebra of the observables then how to connect this idea theoretical idea to the experimental thing is the key is expectation value so theory provides the expectation value of something and we can connect the theory and the experiments by comparing the theoretical expectation value and the average of samples so all the quantitative science depends on this structure like algebra of the observables and some mapping from the observables algebra to the sum number system so this is called the state is in mathematics and the algebra and state state is something like the mapping of quantities of random variables of observables into its expectation value so this mapping of expectation is technically called state so if you start from observable algebra and state this is the starting point of non commutative probability theory that is algebra is commutative then the conventional probability theory can reconstruct from this commutative algebra and state but if this is non commutative as now pointed out if the process is essential then this algebra should be non commutative then we have the non commutative probability space which I called intuitively counter mathematics so in this case the observable dependence is described the other kind of observable dependence can be described because you can't have the classical probability space all the observable but if you choose one observable like position or momentum you can choose this then you can construct the classical probability space so the choice of context or the choice of measuring something by choosing something so context you can have the classical kind of probability so this kind of observable dependence is well formulated by using non commutative probability space so I think these multi-layered observable dependence can be described by using non commutative probability and the last thing I would say is the connection of the category theory and non commutative probability very interesting if the category is given like a causal system give us some category like causaliation is arrow and the composition is A is cause or B because C then indirectly of course at least the A cause C or something so it can be considered as a category there are many kinds of categories but if the category is given then the algebra reflecting this category is necessarily non commutative if there is a non-trivial arrow between two objects so actually the category theory and non commutative probability is from the mathematical point of view it's kind of the two aspects of the same thing like the fundamental observable dependence or contextual dependence or something and this contextuality in topos theory actually this choosing concepts and the transformation between the context can be formulated in terms of topos which is a very powerful kind of category sorry for the too long explanation but without that this very fundamental question will not be organized sorry for the long explanation it's awesome I mean empirically if you have a sensor that's getting measurements at a certain time scale then there's a binding of the time scale of the observer and the processing can I ask a question from the chat alright Dave asks at one time professor Tanini was representing the shape of the relations between complexity and consciousness such that two richly connected systems are non-conscious they're overwhelmed by the complexity recently the inverted U that shows this relation seems to have become scarce leading to undue confusion are IIT people abandoned the idea that there is a bounded sweet zone for consciousness can a system be either too simple or too complex to act consciously so is IIT lead us to a kind of monotonic consciousness concept or could there be a U shaped relationship with information integration and qualia I can also see the comment here so in the end the question is can the system be either too simple or too complex to act consciously in the sense from an IIT point of view there are several things that needs to be said but inverted U is still there in a sense and here this person seems to be confusing between the axis of the complexity and the axis of the connectedness in IIT original formulation and it's still the true for the IIT 4.0 which is the latest one when the system is too connected let's say 100 neurons and each of them are connected to everybody and in a uniform way then IIT predicts that it doesn't have much complexity or compositionality or uniqueness to arise from that kind of system and even if you cut the system it doesn't matter much it leads to most likely a very low level of consciousness I think that's what yes I should have typed connectivity yes if that's the case that's fine but higher complexity which is granted by some small world or some sparse connectivity structure higher complexity is monotonically created with more consciousness that's not the case in IIT and that is true from the beginning and the easiest kind of example is to have a two highly complex kind of system that is loosely connected between them it's a kind of a situation where you are interested in the sense of ant colony even though it looks extremely small the number of the neurons they have is more than 1 million in that order yes around a million so IIT would naturally predict that the causal connection or integrated information arising from this 1 million neurons within an ant are going to be really massive it's very difficult to disconnect in any way however interaction between a single ant to the another ant has a much less direct causal relationship therefore the local minima or local maxima of the 5 values is going to be very huge within each ant but when you combine or when you treat all these lots of ants it definitely does bring some kind of intelligent and complex behavior but phenomenality is not there at that level phenomenality, consciousness within this maximum level of integrated information pretty much like what we talked about in the time scale stuff we have a huge complex and very intelligent behavior and artifacts and also short time scale but what matters for phenomenology is this particular several orders of 100 milliseconds that corresponds to the maximum of the integrated information so you still have emergent intelligence at the colony level but that doesn't necessarily mean that you have qualia arising at that level for analogous reasons why our qualia are at the spiking time scale despite that being only one of the nested time scales exactly IIT predicts that intelligence and consciousness qualia is completely dissociable and that leads to the commentary about the large language model probably the large language model is probably structurally built to be very simple almost like probably in the cerebellum in the human brain which accounts roughly like 80% of our neurons in the brain out of 10 to 11 neurons in the brain 80% are in the cerebellum but even if you remove it or lose it or bone without it consciousness is not that different and that's a simple kind of prediction that the cerebellum is doing maybe something similar to the large language model is doing it's very modular, it's parallel it's not integrated, it does a lot of intelligent things but nothing to do with consciousness very interesting well maybe in our closing sections of course anything you want to share or ask but how does active inference benefit and grow from what you're sharing and how do you see things unfolding for people interested in these topics I have to do you want to say something first? I'm thinking how would you know then from my I used to have difficulty thinking about active inference or things like that but by broadening my perspective becoming more generous I started to feel maybe some aspects of thinking and formulation can be relevant at least for the explanation of the biological systems in various domains and if it's also useful to explain intelligence or construct intelligence that would be great and from our work of the quantum quality of hypothesis, if quality is something that is influenced by its measurement then that aspect maybe emphasizing this usually you know quality of consciousness is considered a passive construct at a given moment what I'm experiencing static passive but now I'm more thinking about because of this collaboration with the Hayat and other people that can be the case and it's more to do with the process it may not be that kind of level of active inference that you are thinking not the level of acting on the word but more to do with this state itself influencing itself might be some kind of important insight to understand the quality ok my comment is ok so it is related to the concept of states again because expectation value I pointed out the importance of expectation value but for example the strength of causality or something then the problem of time scale is very important so you can cause grain for example brain activity for one time scale another time scale so I think if IIT is correct there is a good time scale which is of importance of choosing this structure then I think the brain information or something causal structure should be the good category and you can construct non-commitative algebra and state is defined or something so the time scale is quite fundamental and on the other hand the state on the category that is state on the category algebra can be represented by some kind of weight on arrows the positive kind of weight on arrows so the change in the state is the change in the weight of these arrows the positive definite function I think the transition from this arrow is very from at least mathematically very interesting to compare the active inference as a mathematical point of view so the good weight to the better weight or something is very how to say interesting to compare I'm not I don't have much knowledge about active inference but it seems to me it is quite useful to connect this level the state change or something so yeah it is very interesting for me and the other thing is I forgot the third point but yeah and yes as now already okay points out I talked about the measurements, the instruments but the point of this notion is the probe system so measurement always often considered as a very direct kind of interaction like the system and the environment but environment has some kind of probe into the system then the interaction is occurred with this probe and this system I think it is very analogous to the prauria, the case of so of course outer world is relating to the brain inside of course at least a process or the history or development but I think through perception or something the brain has its probe of the world for example how to say, interesting so measurement, the analogy to the measurement theory may be that some physically within the brain but some kind of probe of the outer system and this prauria is interacting so I think it unifies the picture I think maybe what do you think that's actually very much related to what I was actually constructing this morning so I was reading this Ozawa paper that described the instrument and now I'm getting better understanding what it means what he says is that every observable A of a system S is identified with the observable A tensored with the instrument I of a system S plus S prime so S here is the original system and S prime is the one that with any system S prime external to S and what Hayato was meaning is that this S prime is a probe system which is a part of the environment and so here what this quantum in this case quantum kind of system description but it's also true for the quantum cognition and in my case it is also true for the quantum quality hypothesis is that some kind of quality experiencing system is interacting with the world but through sort of membrane or sort of that like a blanket like thing which is already filtering as a sensory input and attention and this part itself is kind of interacting with itself these arrows are the critical part and then if we translate that Ozawa's way to this corresponds to every coil A of a quantum expensive system S which is in the case of IAT it's called complex Ozawa A times instrument of a system S plus S prime and this S prime here I think will include all the neural processes that supports and underlies quantum but it doesn't really directly include the external environment many people start to do that but if you do that then suddenly hallucination dreaming and things like that becomes very difficult to explain including too much I think that corresponds to what you wanted to say exactly well this is really awesome I very much appreciate that we could have this conversation is there anything else you want to add or ask? thank you I also enjoyed and it was also good for me to formulate myself as well yeah I'm very surprised surprised me in the how consistent our thoughts are how to say it was just a surprise for me maybe the conversation often goes how to say diverse but our conversation has a very surprisingly conversion I think it's quite interesting it's I learned much from this conversation thank you for inviting us thank you so much hope to join on your channel and hear updates on your research as things develop so thank you again thank you bye bye


Having you mean it's when you are so age 5 minutes When you're missing 15 Okay Okay, so we're not gonna see it here, so I'm gonna go there. Yep the keys work. It's just doesn't play Oh Okay, so we have Mitri Marshall and Marcus sigla signal right They're gonna be talking about the portable graphics abstraction in rust So hello everyone, I'm Smith from also a graphics engineer at Mozilla I work in Firefox Rembrandt there and before that I worked in a game-deaf industry Which inspired me to start writing a graphics abstraction layer in breast And this is Marcus Who joined us recently to rewrite it and that's what we are gonna talk about The idea is to have a single abstraction layer To run on all the platforms that we have Unfortunately, the graphics API have been branched out. We have Microsoft Pushing the directorates 12 in the Windows ecosystem, and we have Apple pushing for metal Everything else is Adopting Vulkan quite successfully including the Android the most popular operating system And OpenGL is still alive as well And we want all of those to be available through a single interface in rust that is hopefully safe and fast It started in 2014 and The legend is we had three threads at the beginning and then we were fast-tracking through the evolution of the APIs As the native APIs did so first we got command buffers We got rid of the threads and then we followed with resource views pipeline states Eventually getting the display 11 and metal backends and then somewhere around 2017 when we started first talks about integrating into web render it appeared to us that Even though we already had the core layer split out of the Higher level rendering we still weren't zero cost enough. So we decided to go lower level We decided to implement the core to be completely zero cost and that's where Marcus comes in this is the main Roadmap of Our work and if you are taking one thing from our top, then that should be this slide. I believe At the center you see hardware abstraction layer. That's our new core. That's the name of the new core. It's Vulkan like rust API very low level quite unsafe That different graphics backends implement including Vulkan metal D312 and open GL These are the backends that we already have now and We are getting D3D11 soon enough as well The clients of the hardware abstraction layer are many aside from the user apps We have the Vulkan portability layer, which we are talking about a little bit later We have the warden test framework to ensure no regressions in the graphics tests and we have a web GPU prototype Based on servo that's using that a little bit more about the warden test framework. It consists of three parts First one is we are able to describe graphics resources and passes in readable Source which we use the rusty object notation for It says here's my images buffers Here's what I'm gonna do this with them in the following passes and then we have we can Load it all with a particular graphics back end so that it instantiates the resources on the GPU and creates the command buffers to implement passes and Then we have a ref test framework which has some very simple ref test defined and It runs the graphics passes on the specific back ends compares the expectations with the tests This is an example of the resources. We have an image With a specified format and dimensions. We have a rendered pass With one attachment and one sub pass Just as an example of roughly how it looks This is the contents of the pass. We have the setting the viewports the pipeline and drawing a single single triangle here Okay, so then why we want to actually use rust instead of for example C++ There are a few nice bits of rust and compared to the basic concurrency all const new features like pattern matching and their sensing which basically Tells you that every object in our API is sensing so you can access it from all threats and concurrently and There's also in in mutability, which we use to say for example, you can Change this object or you can't and it also encodes if you need to externally and force Single access because for example some objects can only be Access on one thread or at one time. So if you have if you see mutable access, you usually indicates that you need to External synchronize your object then Traits and associated types, which we use quite heavily which is each you will see later on and We split all the things into multiple repositories For example one repository is the hardware exception layer and then for each back end We have an additional repository and for it. Yeah, okay great So what we are currently missing is for example non exhaustive enums Which are basically required as we also provide an higher level abstraction for this Vulkan portability implementation and there are cases where Vulkan could return us Enum values, which we not really know at the current time of implementing which could come from a Future driver, which would return us this new Enum values then struct alignments and shaders which It happens every day that we misaligned some shader struct because and there's one page of Vulkan specification which basically says how you need to align your fields and it's usually You do something wrong there then const generics, which would give us the possibility to Use areas fixed-size areas more More often in the library interface. We basically expose a quite generic interface using into iterator, but Yeah, fixed-sized areas would help quite a lot for the user side to looks a bit better and Exclusive cargo futures at the moment. You could something could do something like I want DX12 and Vulkan to Run at the same time somehow, but it usually will mess up here and you usually only want DX12 or Vulkan So the back-end trite is our main interface. It's one large trite You have to have to implement if you want to implement a back-end we basically divided into dispatchable objects like the device or the command queue which You can call functions on and the basic resources like shaders Buffer image, which you usually just pass around and create and delete so for the roadmap 2018 we would like to become a Mozilla project because we want to integrate into web render and also into the web GPU part Which will Demetri will talk about and One side project is work on possibility. It's an initiative from Vulkan and from Kronos where they try to Do all the three three back-ends Vulkan DX12 and metal and Provide one RPE subsets from the Vulkan API, which can map to all these three back-ends and Our goal is to implement the Vulkan portability layer using our hardware extraction layer and we currently can already pass a few examples like triangles or pipeline example from Common samples from Vulkan samples and we are aiming at Implementing or hooking up the Vulkan conformance test Yeah, we can we try to hook up the Vulkan conformance test suit and Run run the tests against our hardware exception layer And that's how it looks from our side We basically have our hardware exception layer on the bottom, which is the DFRS side and on the top We have the portability layer and you can Interface it using driver In the implementation or using a static library and if you want to directly link it link it from a C library So a little bit more information about what WebGPU is It's a W3C working group which Microsoft, Apple Mozilla and Google are part of and as well as a lot of independent actors that is trying to design a new API To provide newer lower-level concepts onto the web for faster interactive applications on the web and There's been a lot of discussions over what the next gen API is gonna be and we I'm a part of the group and we don't seem to have a lot to agree on so far Everyone is sort of pushing in different direction So our position there is to have something either exactly like Vulkan or very close to Vulkan and Based on that we developed a prototype that is using servo, but adding in you web-idl based API That has basically Vulkan calls in it so you can go that low-level and create your pipeline barriers and descriptors on the web using JavaScript or WebAssembly and Run it in the browsers which in the browser which is currently a very specific fork of servo Our future plans with the WebGPU include trying to get a native application Trans compiling to the web using WebAssembly So a native application using Vulkan to be compiled using WebAssembly for the web on the browser So that they just trans in the browser so that you don't need to learn a new API that you can just use Vulkan on the web Now a little bit more about the ecosystem that we have and since we seems to have a lot more time left that we're gonna stop on Each of them closely There is a few engines in Rust at the moment. None of them is major enough But they're very promising namely good game easy amethyst and 3rs All of those use the older Jeff XRS at the moment where Amethyst is already transitioning towards the new hardware abstraction layer It's extremely promising to us is they not only Are porting to the hardware abstraction layer? They're bringing bringing the higher levels for us to use for example a Library that manages memory for you so that you don't have to care about Particular memory types that the driver exposes you just say oh, I want this buffer and the library handles the actual heap management and allocation of the memory from the driver for you or a frame graph sort of thing which allows you to describe all Your graphics workload as one big graph Which is then automatically deriving the dependencies the memory barriers for you and Possibly schedules it to run on multiple hardware cues for execution in parallel So this is what amethyst folks are bringing on the table, and I'm sure a good game easy and 3rs will follow and use hardware abstraction layer in 2018 other than that we have a very interesting plans with volcano as a Project that we are often being compared to in fact now the comparison doesn't make much sense because we are lower level We are implementing Vulkan and Volcano is based on Vulkan. So what we want to do is to Let users use Volcano on top of gfx hardware abstraction layer So in a sense, it's a drop-in replacement. The users will not need to do anything and they will instantly get access to Apple and Microsoft platforms potentially Xbox one as well So that's something that we are working on right now and finally web render is Mozilla research project that is currently being integrated into firefox it is in need of better control of the latency resources and More direct access on Windows platforms for for example playing a video efficiently so our Vulkan portability or graphic abstraction layer makes perfect sense to be a part of web render and We are already working on that. We have a group in Zegat University in Hungary That's 14 by Brenda and they have an exciting progress so far so they translate the shaders automatically and They already rust they already run our ref test framework. So we're looking forward to see web render running on how this year potentially upstreamed as well to Firefox and We have a very lively Gitter chat, which does not have a link here But we have also a blog post and you can just jump on the project page on github And it will have all the links that you've you will ever need We are always happy to answer your questions and right now I'm going to run a demo that Marcus have reading written the demo features quite a few compute shaders and some rendering as well and It's fairly complex it simulates ocean water and it runs Exactly the same on D3D12 metal and Vulkan using the same shaders and the same API calls So that is it Ready for questions So the question is why do we consider it a zero cost abstraction layer? It's zero cost in a sense that if you want to It to be backed by Vulkan then we don't do any Hip allocations and we don't do any smart pointers Management for you. It has explicit like allocation and free calls for resources Of course There is certain warts that we have to do in order for the same API to run on D3D12 and Metal so this is not exactly zero cost in that sense This is as low cost as we can afford but this is this goes in line not in our own reasoning but more in line of the Vulkan portability Technical subgroup and the design that we're doing as a part of the group. So It's very little minimum overhead. We can't afford any Extra GPU work for whatever we are doing Is your question that we are losing quite a bit of command buffers in metal? Okay, so the question is how do we deal with the performance loss from not being able to reuse command buffers in metal? that is very unfortunate and what Molten VK is doing as one of our competitors technically is Having a like a software list of commands build up independent of the hardware list. So I Don't think we have that implemented right now, but that's the plan right now Are there any plans for the graphical UI toolkit? We are extremely busy with making sure that the hardware obstruction layer like the lowest level of obstruction works and our next steps are more in terms of Getting the like frame graphs or the flibries or the memory management libraries solid before we can think about the UI however, I expect Conroad to be able to be ported on JFX hull eventually. It's a graphical UI toolkit for rust as well as Our friendly amethyst folks are likely to come up with something too. So we hope for your help So historically before that you write we required we required sorry the question was how do we handle shaders with regards to different platforms? Historically we Required the users to provide all the shaders for the back ends that they use like separate shaders for metal OpenGL and D311 after it the real right as we are going for the Vulkan portability goal we only accept SPRV and From SPRV we generate all the back end specific shaders We patched the logic like coordinate systems and descriptors IDs according to the specifics of the back end that we are Translating the shader into We're using the SPRV cross as like a helper library to do that and our far future far plans would be to Have a rust alternative to that as well since that's the biggest non-rust component that we use right now So right now you supply your shaders in SPRV. Of course the most convenient way to do that would be to write it in GLSL Say 4.1 and have them convert it using GLSL to SPRV That's what Warden test framework is doing at least you write your shaders in like GLSL and test them against SPRV so the question the question is how does webGPU is going to expose Vulkan given that webGL is way higher level so yes, it's a hard question and Vulkan is a hard sell for multiple reasons on the web Not only because it's low level, but all the consequences from it is like it's easy to build a user signature based on What you query about what device can do and on the web it's not safe And there's a whole lot of concerns that not only make not only Vulkan But in general going any low level on the graphic side To slightly start breaking the rules of the web. So we're trying to find our philosophy there and Our priority is getting like the maximum performance of the web


So my name is Lauren obviously and I'm a librarian and I also am completely in love with WordPress Which is why I'm here. So thank you all everybody who's like contributed to making that happen And I'm going to talk today about teaching the public in the public library how to use WordPress to make their own websites So why the library right so my library's mission is really broad Basically you can fit anything in it. It's to contribute to the recreational cultural and educational needs of the people of Queens Which is everything so educational, you know WordPress works for that Libraries in Queens are everywhere if you are in Queens anywhere other than JFK you're in walking distance to a public library I work at the one in Jamaica and it's the central library. And so we have a great training room And it's a really just a good space to teach We're also everything's free. No one charges for anything. We're trusted in the community and We provide services for everybody and it's a just a great place to do that also Libraries are the old-school open source. We've been lending things out for free since 1732. Thank you, Ben and We haven't really stopped since then and so when I'm teaching something I'm usually looking for something that people can access at home for free And WordPress does that for them, which is really fantastic. So That's why WordPress so I could use there are so many different ways to build a website that are web-based That we could use to teach people how to make a website in the public library But we use WordPress because well, it's open source. It's free It has a community versus being locked into paid services and as a librarian I'm super uncomfortable, you know telling someone they should use a particular commercial service That doesn't really work for me and Telling someone that they have to use a service that they have to pay for also doesn't work for me because I work in a really economically diverse community Some people have money and a lot of people really don't and maybe they want to have a blog anyway Maybe they want to have a website and start a small business anyway But they don't have a lot of resources to do that WordPress fills that and it's really well So the most difficult part of doing this has been the technology setup Like hands down that has been the most complicated thing. So I've tried a couple different ways to do it The first the first time I tried a one-session workshop with wordpress.com It has sort of worked except that no one ever remembers their email password ever Ever remembers their email password like two times a day. I have to help somebody recover it saying And so like you can't get the confirmation email for wordpress.com because you can't remember your email Fastword and then that takes up a half an hour of the class just recovering everyone's password So that didn't work Then we used instant wordpress Which was good, but difficult to get the sites off as in difficult as in I haven't figured it out yet So those practice sites are still sitting on those flash drives. It was really great. I Also didn't have a budget for flash drives So it wasn't really sustainable who used the lost and found ones that have been left there for like a year cleared them out You know people if you leave your flash drive at the library for a year We're gonna assume that you don't need it anymore And then we did the thing that did work Which was I have a set of Mac books that we got from a grant a couple years ago where somebody said spend this money And I said I want Mac books and they gave me Mac books that it was magical And we installed a map stack on each of them to have a test environment for people to try it out and So there's a reason that I'm doing it this way specifically So I want to make sure that the people who are learning how to build these websites have enough of an understanding of how a website works Underneath that they can talk to tech support when it inevitably gets screwed up right and so Building the map stack and having them build it on the org side instead of just doing the comm besides the email portion of it Means that they have to learn the vocabulary and they have to have sort of an Understanding of how it works so that they can do that next time when they go to do it So we break the workshop down into three parts Each one's about two to three hours long depending on how much time I can get the room for The first workshop goes is set up and wordpress structure, and then we talk about servers How a server stack works what it is what the vocabulary and languages and This is where I lose people Which is actually fine? So a lot of times people will show up to the class thinking it's Microsoft Word And or thinking they can just Hop in and do it and they're not ready like they've just come off of intro to internet We just taught them what a browser is they're still figuring out the mouse and they want to do a website real bad But they're not quite there yet, and they need more other classes to prepare them for this And so we put the complicated scary stuff up front so that we can then have a conversation with them about What they need to do to get to the point where they are ready to do this part So in this class we up we do the setup the server structure, and then we also do How to do wireframes and how websites are laid out In class 2 we talk about how to add content the difference between pages and posts and The difference between plugins and widgets because nobody understands that either Everybody thinks they're the same thing Even though they're totally different and not related. It's fun How to find images and how to add them to the media library and manage that and then we talk about image size Which is also really important because they will put huge images in there, and then it will take forever to load In the third class we wrap up everything we did in the second one take a look at what we've got on our test sites And then we talk about accessibility security and SEO and a lot of the accessibility stuff We go through the tools that you can use to check we go through Talk about adding alt tags and what a meaningful alt tag would be Because a lot of times if you've never built a site before and you've never used an alt tag You have no idea what it is and these are almost always very very beginners and then for security would talk a lot about passwords and Making sure it's not the same that you don't use the admin as your admin name and all of that stuff Lots of stuff doesn't work So the very first time I did a five session or well, it was supposed to be six six session workshop where we were Having teenagers build WordPress sites for local small businesses One we had a giant blizzard right before the very first class And so I had to call from my house everybody who was on the registration list and be like don't drive You know don't come to the library and then The businesses didn't participate the way they were Supposed to so they didn't show up which makes it difficult to teach a group of teenagers how to talk to small businesses about what Their site should look like if the small business doesn't come And so we have a really fantastic laundromat site kind of hanging out just waiting for the laundromat to claim it You know And then instant WordPress didn't quite work out the way I wanted it to either because we couldn't move managed to move the Sites off it I did have one business who came in the beginning and then came at the end to get the site And we actually had to rebuild it on his hosting service because we couldn't get it to move over But even with all that we had a lot of really great results We've got two blogs that are set up and running that I'd know the people and still talk to Alexander's mom calm and big it the fitting style calm, which are really great One woman had been a web developer Before but hadn't worked with WordPress very much And she actually started getting job interviews because she could transfer her knowledge over and now She had the vocabulary to talk about WordPress So she got at least one job interview, which was awesome and Then one of the great things about it is that teaching multi-session workshops in a library People don't show up for the rest of them They come in for the first one and the second one and maybe they're like, yeah, it's free Why come back? but this one we managed to hold on to almost everybody and that was Really great We also got really fantastic survey results For everybody and the super fun laundromat website, which I'm still sort of hanging on to Basically, I think we need to think of libraries as a place where people can learn things like this We do a lot of basic digital literacy and teaching people how to use Microsoft Office and teaching them how to find job Like use that to find jobs in regular industry, but we don't always bring in the advanced things for adults We'll do coding for kids because it's trendy and that's what libraries, you know Do we do education and everybody's talking about that? And so we bring it in but we really do need to be reaching out to adults and teaching them career skills that they can use that are more advanced than just Microsoft Word and WordPress gave us a really great place to sort of fit that in without having to teach coding itself Which is really difficult to do with people who are just in that intermediate digital literacy stage like they've got Excel But getting decoding is hard so I Really hope that a lot of people will try this and we'll try to Bring these sorts of things into libraries because I think it can be a really big difference in people's experiences I'm good


So I got a message the other day from someone who I'm not going to say the name of And they were just asking me. I think they would be nice. They just said, you know, how come you never edit your videos? Why do you why do you not edit your videos professionally like with iMovie or you know? Some sort of movie editing program and there's a fairly good reason for this So the first well a few reasons actually the first reason is I've watched a lot of youtubers that they spend hours and hours Editing their videos And as a result, they'll maybe put out one a week or maybe even once a month sometimes And the video will be really good. It'll be really high quality But it will just be once a month and I feel like there's a sort of There's sort of a trade-off between having really high quality videos and also posting regularly and keeping in contact with the audience So I think if you can if you do have the time to edit every single video and post one every day or one every few days That's great, but for me personally. I like to Not edit my videos so that I can post more now I know what some of you are gonna say you're gonna say well, how come you don't post regularly then There's a few reasons for that but I now have a lot more time available And I'm gonna be doing hopefully a video every day Monday to Friday So that answers that Second reason is I like watching videos where and you guys might be different Maybe this is just me, but I like watching videos where it's very natural and there are natural pauses in what the person is saying So what I mean by that is you'll notice that sometimes in my videos I'll stop for a few seconds and I'll think about what I'm saying or I'll I'll pause in between different ideas and different thoughts To give myself a time to think about what I'm gonna say Because these aren't scripted videos remember And also to give you guys a chance to sort of let what I say sink in so that you're not just being bombarded with Information and you just constantly You know having thoughts thrown at you because a lot of these YouTube edit youtubers that edit their videos They tend to sort of cut things so finely that you don't have Even a split second to hear what they've said it's sort of like one thing after another and they'll edit the video so that they The next part of their video starts talking or starts making noise halfway through the last part And it's it's very hard. I find sometimes hard to keep up with what they're saying So I like to keep it a lot more natural unscripted and more regular Let me know if you actually like this. Let me know if you want me to maybe do a mix or do some more edited Maybe have some animations. I did some whiteboard videos a while ago, but they sort of no one really watched those There was a problem with recording the audio. It wouldn't sync up for some reason there was like There was a big delay between what I said and what was recorded and what What was shown on the whiteboard so Maybe that's something I could look into if you guys are interested in that but till then I'll just keep doing these videos on my laptop and Answering your questions. So there's gonna be a daily video Monday to Friday answering questions There'll also hopefully be one every week or two going through like a major technique a review Or something like that side note. I'm gonna be doing a full review soon of this Now for those of you who don't know this is the sleep shepherd Which is a lucid dreaming Induction mask, I would say although it's not marketed as that because it's marketed as a way to improve your sleep and Listen to binaural beats while combining that with this thing which monitors your brainwaves and it sort of alters But anyway, I'm gonna go into that in my next video. So I'll give you a little tease of that one there so make sure you're subscribed and Look out for that in the next few days