Hey friends, welcome to Divine Intimacy Radio, behind the scenes, and we're talking about spiritual warfare today and the lights started to dim and I'm like, what is going on here? Anyway, behind the scenes, Divine Intimacy Radio, today we're going to start a series on my upcoming book, which is called Devil in the Castle, St. Teresa of Abulah, Spiritual Warfare and the Progress of the Soul. And I think it's my favorite book that I've written, and not every book is my favorite book when I write them. Some are just, I write them for certain reasons and are helpful, whatever, but this one I really think is very powerful. The, if you want to get a pre-ordered discount, so as always, you know, you're going to find, I think all my books are at EWTN Religious Catalog, but it's not out yet. So as of the recording of this show, today is January 11th, 2022. But if you want to pre-order it, you can get it from Sophia Institute Press.  The other thing that Sophia is doing to help folks use this book is like a study group, because it has questions at the end of every chapter and a summary of the chapters. It's very, very, I wrote it very much oriented to small group discussion and very practical, is Sophia is offering, I think somewhere in the neighborhood of a 50% discount for bookspot in a case, which is a pretty significant discount if you're looking at doing it at your parish or whatever. So, but as always, EWTN's Religious Catalog, we should support EWTN because when you buy books from EWTN, two things happen. They send them to you and you help the ministry of EWTN. Okay, you ready to roll? Okay, and your mark gets set. This is Dan of Stephanie Burke. Welcome to Divine Intimacy Radio, your radio haven of rest. Your hermitage of the heart. Your monastery of the mind where we lift our hearts and minds to heaven to draw upon the wisdom of the saints. And today, we're going to be drawing on the wisdom of a doctor of the church, a doctor of prayer, a woman who's written probably the most important book in the history of the church on prayer. And that's St. Teresa of Abulah. A beautiful woman, a strong woman. Yeah. She's not a wallflower. No, she was, man, warrior for Jesus. Wow. Amazing woman. But the book is called Devil in the Castle, St. Teresa of Abulah, Spiritual Warfare and the Progress of the Soul. And we're going to be launching a series, actually, we are, that was really bad English. We are launching a series right now, right now on on the book. We're going to walk through it chapter by chapter. So one thing I will tell you, the the forward is written by Dr. Anthony Lillis in it. And in it, he does a beautiful job of setting up and explaining the analogy of the interior castle, which the book is based on. Right. What struck you about that analogy? I know you really loved, I know you love his writing in general. Yeah, the history and understanding the context of why she's writing this book, why is it's important, why it's important, the impact it had. Yeah. And it's just an extraordinary book, especially for that time. But it's, it's a classic for any and all, all time, who desire to grow in the interior life, right? Yeah. It's a beautiful roadmap. It's super inspiring. I read it years and years ago. I've picked it up again, time and again, and it's just beautiful. Yeah. And if folks have had trouble understanding her and her writings, this book actually will help them because it extracts certain ideas. And if you read it side by side, it will illumine things that might be hard to understand. Yeah. I think that's a great idea to do that. So the reason I named it Devil in the Castle is because the devil is in the castle. Yes. The castle is, of course, an analogy for our progress toward God, and that St. Teresa of Avila gave us in the interior castle. And the book is about how we progress in the spiritual life and how at each stage of our progress, there are different kinds of tactics that the enemy uses to get us off track. So I don't know, have you experienced any of that? Just a smidge. Yeah. Yeah. But the cool thing is, all you have to do is persevere and you win, right? Persevere in the sacraments, persevere in prayer, persevere in the, you know, following Jesus and you win. Yeah. But we can do far more than just persevere. Right. Right. We can, we can really live a life that even though we have these knits, you know, nipping at us, we can, we can find our way and in a very profound way to come to know Jesus in a, in a transforming way for us. Yeah. I mean, some of us, some could look at the title of this book and say, you know, that kind of scares me. You know, I don't, I don't want to really look at these things, but, you know, I would encourage you because it's very hopeful, right? It's hopeful in that when we understand the progress of the soul, understand how the enemy seeks to derail that, but also understand the beautiful gifts, graces, tools that were provided through, like we have everything we need in and within the church, through the church, all the sacraments and everything, plus these beautiful writings of a doctor of the church, such as Teresa, then we can rest easy because how can we possibly progress if we don't know where we're going, how we're going to get there? You know, otherwise we just kind of float around. And I think one of the, when we talk about the, you know, the forward from Dr. Lillis, one of the important things he says that I found so important to understand is that contemplative prayer without theology is dangerous, right? You don't want to go into these waters, kind of reaching around without understanding, you know, what are the teachings of the church? How can I navigate this? And these, the books that you have written, one of your giftings is really helping people to understand how to progress in this interior life, not get lost and not go reaching out into Eastern whatever to try to find answers. So everything's there, right? Yeah, yeah. Good. Well, of course, as the book, as Teresa proposes, and as I propose in the book, the devil is active in your prayer life, in your parish, where you attend mass and the lives of actions of people of good will around you.  And if you know how he works, you'll know how to fight. And if you know how to fight, you will win. And if you don't know how he works, you will be defeated. And if you don't fight, fight, you'll be defeated, and you don't get to dodge this either. If you don't, if you think you're dodging it, that's just the enemy convincing you of a strategy he really likes. And you're going to end up losing the ultimate battle at the end of your life, and which is not good. And I'm speaking, of course, through the lens of tradition. So Teresa of Abel is a great mystic and guide to this, this great battle. I think in my introduction, I give several examples of both, I think most of them are outside of the interior castle from her other writings of her direct experience with demons. Do you want to, since you are a woman and she is a woman, you mentioned it, be good for you to read it on page four, which is about her experience with demons and mass, right? Absolutely. You know, she says in one, well, you say in one instance, Teresa describes how she turns cold and tense as she comes face to face with the eyes of two demons. So here's her account of that. And I beheld my Lord in that great majesty of which I have spoken, held in the hands of that priest in the host he was about to give to me. It was plain that those hands were those of a sinner. And I felt that, and I felt that the soul of that priest was in mortal sin. What must it be, oh my Lord, to look upon thy beauty amid shapes so hideous? The two devils were so frightened and cowed in thy presence that they seemed as if they would have willingly run away had thou but given them leave. So troubled was I by the vision that I knew not how I could go to communion. I was also in great fear for I thought if the vision was from a God that his majesty would not have allowed me to see the evil state of that soul. So this comes out of her life, which is her autobiography. And the Lord goes on after that vision to tell her why he revealed it to her. And so he revealed it to her, interesting for a purpose to teach her that the Eucharist is and is not or the Holy Sacrifice Mass is not made invalid by the sinfulness of the priests, which of course was a longstanding doctrine of the church prior to this, which was mainly enunciated by St. Augustine in the fourth century, when he was writing his treatises against the Donatus, I think it was against the Donatus, but or his treatise against the Donatus. But essentially, it's just saying the Mass is priest-proof in the sense that the state of the soul of the priest doesn't impact the Mass. But nonetheless, you know, she's this woman is not a weak woman. This woman is a woman of deep prayer. And by the time she writes this book, this is this book was written at the near the end of her life. This is at the peak of her understanding of all that is good, true and holy and who God is. She's very close to the Lord and let she's terrified by seeing actual demons, two of them around the neck of a priest as he's offering Mass. Right, right. And she says that the Lord himself tells her to pray for the priest, right? And what I find, I guess, edifying in this is her description of how they cowed being in the presence of the Lord. Yeah, they were trembling because of the Lord's presence in the Eucharist. Right. And they, you know, and just from our experience in exorcism ministry, we know that similarly, or in many of the exorcisms we've been in, we have the holy sacrifice of the Mass with the victim or the inergament as before we go into the exorcism, which strengthens, which weakens the enemy quite dramatically. Right, right. Well, it's a beautiful testimony to the reality of the body, blood, soul and divinity. Yeah. In the Eucharist, right? Yeah. The power of that. Fascinating stuff. There's another vision of the encounter of her that the Lord revealed to her regarding the forces of hell and the power they hold over a soul who fails to live in a state of grace. I think we have time before the break for you to read that one. Okay. At another time, I had a vision of a different kind, which frightened me to the core. I was in a place where a certain person died who, as I understood, had led a very bad life and that for many years, but he had been ill for two years and in some respects seemed to have reformed. He died without confession. When the body had been wrapped up in the winding sheet, I saw it laid hold of by a multitude of devils who seemed to toss it to and fro and also to treat it with great cruelty. I was terrified at the sight for they dragged it about with great hooks. But when I saw it carried to the grave with all the respect and ceremoniousness common to all, I began to think of the goodness of God who would not allow that person to be dishonored, but would have the fact of his being, his enemy concealed. So just to pause there, what she's saying is that she saw that this guy's going to hell. He didn't settle up with confession before he died. So she saw the demons kind of rejoicing and getting ready to take him to hell. But the Lord protected his dignity in that that was not revealed at a particular portion during the funeral. And then she goes on after that. During the whole of the funeral service, I did not see one of the evil spirits. Afterwards, when the body was about to be laid in the grave, so great a multitude of them was there in waiting to receive it, that I was beside myself at the site. And it required no slight courage on my part not to betray my distress. I thought of the treatment which that soul would receive when the devils had such power over the wretched body, would to God that all who live in mortal sin might see what I then saw. It was a fearful sight. It would go, I believe a great way towards making them lead better lives. So let's go to the break and then we'll talk about this when we get back. Okay. Just while we're in the break, I forgot to tell you guys on the same topic, 2022 Avalos Summit is going to be on spiritual warfare by the chair. It's going to be amazing. Yeah, it's going to be at Catholic University of America, Masses at the Basilica, the National Shrine of Immaculate Conception. If you haven't been to that shrine, it's just, you could spend days in there exploring all the chapels and dedicated to different aspects of Mary and apparitions and the life of Mary. Honestly, if anybody come, I mean, the people that come should add on a little bit of time. There are so many riches there just at CUA in the Basilica and the Shrine of JP2, which is just across the street. Amazing. Yeah, it's beautiful stuff. And then there's the Dominican House of Studies, which is pretty neat too. You can visit their chapels, beautiful over there. So it'll be Bishop Strickland, Father Rippiger, Father Inkmanus, Monsignor Rosetti wrote The Diary of an Exorcist, Kathleen Beckman, Stephanie and I. So July 15th through the 17th, you can sign up for that at spiritualdirection.com forward slash events. Last spiritual warfare thing that we're doing is that I'll mention and we'll jump back into the text is setting the captives free, which is about discernment of spirits are going to be two events in one in February in Nebraska and April in Ohio, and they can find out those out those in both at spiritualdirection.com events. So let's jump back into the text to the devil and the castle on your mark get set. Go. This is Dan and Stephanie Burke. Welcome back to Divine Intimacy Radio. We're talking about the devil and the castle spiritual warfare in the progress of the soul, according to St. Teresa of Avila. And before the break, Stephanie was reading an account of a man that the Lord revealed to her died in a state of mortal sin. And I just, it's so disturbing, I have to say, because the way she describes it is, there's just a bunch of demons and I just see them in the bottom of the casket. And as they're lowering the body after the funeral, waiting to receive him. Right, right. Wow. You know, and it's, it's distressing because you understand that he led a very bad life according to, to what she wrote. Yeah. And that through his illness, he got a little bit better, but he never went to confession, went to confession, or he never requested that, or maybe his family didn't call a priest, right? So it's, it's just such an important understanding that the battle at the end is fierce. Yeah. And not only do we need to live a holy life, but we also need to ensure that we have those final graces of, you know, last rites and confession and all of that. We must take care of, of our, our loved ones as they leave this world. Yep. Yep. God be praised. My dad was baptized and confirmed and received into the church and received last rites just before he died. Yeah. That is not common though, you know, the priest who we were dealing with noted like this man in this situation that most people die how they live. Say that again. Most people die the way they live. The trajectory of the soul at death maintains itself.  And that's what my dad responded to. I said to him, you know, you're a good man, but that doesn't solve the problem of sin. Would you like to be forgiven? And he said, yes. Yeah. Yeah. And then so we baptized him. And it's just beautiful. But the trajectory. So what Teresa is trying to do here in this book, well, in the interior castle, and then what I'm trying to do in this book is extract out her wisdom that helps us understand from the very beginning of the first stages of the interior life, meaning when somebody really first begins to say, you know what, I'm going to take this seriously. And they become an authentic disciple of Jesus. That means not just attending mass, but, but really deciding to enter into daily mental prayer, begin to become aware through the examine and that sort of activity where they really are in their spiritual life and stop making their faith secondary or tertiary or whatever. But primary in their life, and they begin to journey toward God, then this book will walk you through her wisdom about, okay, on your first steps into the interior castle, which is what it means to be an authentic disciple of Jesus. And these are the way the demons will work against you. And in it, what I've done is I take, I talk about her references to different demonic deception. And then at the end of the chapter, I summarize the fundamental strategy of the demons, and then how it is that we can fight against it, and then questions for reflection. So it's super practical in that way.  And it's a gift that they have, right? It's, they're not a men tend to compartmentalize and that sort of thing. It allows us to care for children and households and, yeah, do a lot of different. Yeah, right. Yeah. So it is, it is, there's nothing wrong with it. But it can be difficult because she writes Stream of Conscious and doesn't edit her work. She never edited her work. So, so she'll, she'll talk through the book and then go off on a bunny trail and talk through the book. And if you don't know the context of what she's talking about, you don't know the history. And you don't have a lot of time, like I spent in this book to kind of go back and say, okay, now we're back to it's hard to extract these things. But this book lays them out, extracts them in a very systematic way and complements her global thinking with very systematic perspective. So God be praised, we have this great doctor of the church doctor of prayer, who helps us. And what is her motivation? Her motivation is, I want you to know the fullness of Jesus who's at the center of the castle. And I want you to understand that there are ways that you need to, things you need to do to cooperate with God. There are things that you need to do to fight the enemy at each stage of your interior life, so that you can reach the, reach the, the place where you're in union with him. Yeah. And I, you know, you think about, well, why would she take the time to recount these visions that she had? And why would the Lord allow it? But I think it's so important for us to, just as we need to understand that God is real, that he's calling us to this beautiful union and relationship with him, that the enemy is also real, and that his end goal is to derail us and to keep us from getting to the one, to the lover of our soul, who's at the center of that castle, right? That beautiful, faceted diamond that she describes in the interior of cast, interior castle, and that, that the more that we journey towards discovering him, and if we can overcome these different tactics of the enemy, that his line shines all the brighter and then we'll draw others, not only ourselves to him, but draw others to him as well. Amen. Amen. So if you want to understand how to fight, well, there's no better guide than Teresa. She was a fierce warrior for Jesus, really turned the world upside down in her time and then after, and has brought, I don't know how many people to, to, to the Lord. But the one thing I want to mention before we close out the show today is one, just practical, you can find the book at spiritualdirection.com forward slash shop. You can also find it at EWTN's Religious Catalog. There are discounts for bulk purchase available at Sophia and Stupress. But the thing, the thing, what I would strongly encourage you to do is that you first read it, but then also you get a study group around it and read it. The reason I wrote it now, and I didn't write it years later as well, years before I was one, I wasn't ready to, I'm like, like many people, I, when I first cracked the interior castle, I, I sort of, I read maybe 10% of it and then entered into the normal fog that was, that settles in just because I was far too immature spiritually to understand it. But also, it's important because in our time, the spiritual darkness is really, really heavy and it is not going to abate. There's no sign of it abating anytime soon. And there are all, every, all indications are it's getting worse as, as we've talked about before. We, God has given us a sort of a weird life in the sense that we're, we have many exorcists friends and there isn't a single one who would say it's easier now. Every single one and their lay support teams would say it's harder now than ever before. And I think at, at one of our exorcists friends who helped us develop, helping us develop a purely Catholic method of deliverance called Liberazio, he revealed that he, though it's harder, God gave him new means in prayer and in the sessions to free people that he had never availed himself before. They're, they're always there in terms of specific ways of praying to archangels and, and things like that. But similarly, I think this is a time where we need to extract new tools, both in my book, Spirit to Warfare and Disarmament of Spirits, which is attempting to provide an easier front door to the, the wisdom of the great warrior, Saint Ignatius and his wisdom on Disarmament of Spirits. And now this book, which is attempting to illumine this great warrior, Saint Teresa of Avalos Wisdom, so that in this time we can fight and really not be overtaken by the darkness. Right, right.  And one of the things that you've said over the years, and I just think it's, it's a little, it's a little direct, but it's true. And it doesn't, it's not intended to be dismissive of people and their suffering and their mistakes they've made and, you know, the paths they've chosen.  Yeah. It clouds our intellect and our mind, our ability to understand. And you see it, there's like, you know, I, and I don't know who it was that was writing about like almost a mass psychosis that's going on with all the craziness that you see out there.  Right. And, and it's just so difficult. How do you navigate that? How do you get some clarity? Well, the way we do that is, you know, getting some silence in our life, getting some of these influences out of our life, and then engaging in prayer and these, these other means like this book, like your other books on into the deep and spiritual warfare and discernment of spirits and really saying the Lord has provided all these means. I need to study and understand. We must be intentional about our pursuit of God. You know, the, the church through scripture tells us and we see it over and over again to that we need to love God with our whole heart, our whole mind, our whole spirit, all that we are, right? Why does it tell us to do that? You know, for, for what purpose? Because it benefits us because it's necessary. And he deserves it. Right.  We can't pursue God as a hobby or a part-time, you know, thing that we do on the weekends. We must understand that the salvation of our soul, the salvation of those entrusted to us depends on our commitment, perseverance and, and nurturing of the desire that's been placed in our hearts for God and to fight. And we must fight. We must fight in a holy way, but we must fight and we must pursue and we must persevere. And if we don't do those things, we're going to go to hell. I mean, that's just the bottom line. If we're not in, if we're not all in, we're not in it all. And you see that with the encounters of Jesus with, you know, the rich, rich young guy who was reasonably holy and he wouldn't, he wasn't all in, he was parts of him he was attached to, which is how demons control us. And he walked away and Jesus, of course, was sad to lose that soul. And, you know, that's an example of what, what it means if you're not all in, and you not only need to be all in with respect to your goods and the material resources God has given you, you've got to be all in for prayer, all daily mental prayer, all in for the rosary, the exam, living as an authentic disciple of Jesus, not just checking the church box, not just, you know, three dollars worth of God, just enough to make me feel like I'm okay and, you know, all of that.  I have no, there's no pleasure saying that. But the reason I say it is hopefully to shake you up a little bit and get you moving. So I hope you pick up the book, Devil in the Castle, I know it will help you. And I know that you'll learn to fight and you'll help others to do the same. You'll be a light and light the world around you in a very difficult time. Okay. So until next time, may the God of peace make you perfect in holiness. May he preserve you whole and entire spirit, soul and body. Irreproachable at the coming of our Lord Jesus Christ. Amen. Before you take us off, if you haven't yet, we're going to pop out. So we're leaving EWTN's Facebook page for this live stream. You can find us also on YouTube on spiritualdirection.com, YouTube. Make sure you like and subscribe there to help get us, you know, a little bit more traffic to that page. We're, I think we've been shadow banned for a long time now given the decline. But the, we're going to pop over to spiritualdirection.com, Facebook page, live streaming, and we'll also be live streaming out on YouTube, spiritualdirection.com. So to do, we're going to go into, after this, we're going to go into the first chapter. So we covered the intro. We're going to go into the first chapter in the battles that people face, as they're just beginning to enter into the interior castle. Okay. All right. Thanks guys. See you. God bless.


Acetaminophen, APP, hepatotoxicity is caused by extensive oxidative stress, primarily for mitochondrial dysfunction and impaired cell signaling, which can be targeted by therapeutic agents such as mitochondria-targeted antioxidants or repurposed drugs to increase their potential application in patients with APP overdose. This article was authored by Quidu, Anupramachandran, and Hartmut Jaijki.


In the practice of medicine, many fundamental biological pathways that require tight-on, off-control, such as inflammation and circulatory homeostasis, are regulated by serine proteinases, but we rarely consider the unique protease inhibitors that, in turn, regulate these proteases. The serpins are a family of proteins with a shared tertiary structure, whose members largely act as serine protease inhibitors, found in all forms of life, ranging from viruses, bacteria, anarchya to plants and animals. These proteins represent up to 2-10% of proteins in the human blood and are the third most common protein family. This article was authored by Marie-Christine Bautin, Margaret Geiger, William P. Sheffield and others.


I would like to present a few more examples of ocean constructions which are useful in algebraic topology. Before that let me make one remark. Suppose you have a map from Sardvan to Sattor which is a homeomorphism, similarly another homeomorphism from Y1 to Y2. X1 is a subspace of Z1, X2 is a subset of Z2 and under phi X1 goes to X2. So I am doing a junction space constructions over Z1, Y1 and over X1, similarly Z2, Y2 and X2 through maps F1 and F2. I assume that F2 is exactly equal to, you start with X2, come to X1 via phi inverse and then take F1 and then go back Y upside. If you have this data, then what you have is a disjoint union Z1 and Y1, two disjoint unions are two Y2, you have homeomorphism, phi disjoint union of psi, I mean the patch up of these two homomorphism because they are disjoint union. But now because of this relation that F1 is, F2 is precisely F composite psi composite F inverse, this compatibility, whatever you do identifications here, corresponding identifications will take place here. Therefore, under this, the homeomorphism from phi disjoint union of psi, you get a homeomorphism, for example, you get a map which is a bijective from the adjunction space F1 to adjunction space F2, ok. Once this is told, then the verification is totally obvious. First of all, you should see that there is a satiristic map which is bijection. Continuity of this one will be totally obvious because the corresponding functions are continuous on the mother space, ok. So, similarly, if you take inverse is, that map will be from Z2 disjoint in Y2 to Z1 disjoint in Y1, same thing will apply, so inverse will be also continuous. So, this is the topological invariance of adjunction space. Later on, we will see similar version for homotopy invariance of adjunction space which will require less hypothesis than for homeomorphism. In particular, we may say that homeomorphism type of the adjunction spaces depends on the ambient homeomorphism class of the adjoining maps. If the maps F from X2Y or and X1 to Y1 to X2 to Y2, they are just related by homeomorphism, that is not enough. The homeomorphism should take place on the domain at the top level named as Adjunct Z2 and corresponding homeomorphism Y1 to Y2 so that the two maps are correspond under that under those homeomorphism, that is the ambient homeomorphism class. So, here is another construction now which is a slight modification of the mapping, mapping cylinder. Remember that mapping cylinder of a function F from X2Y was defined as quotient space of X cross 0, 1 interval disjoint union Y by the relation that X, 1 is identified with Fx. So, you can think of this as adjunction space of X cross I disjoint union Y via the map which is defined on X cross 1 to Y, so this is a adjunction space, this we have seen. Instead of that, instead of X cross I, you take the cone namely X cross 0 is collapsed to a single point, then X cross X itself is identified as X cross 1 as a subspace. So, you can think of Y as defined on a subspace of this Z Cx and then take the adjunction space that is called the mapping cone of F which is nothing but another way of looking at it is it is the quotient of the mapping cylinder in which the first coordinate X cross 0 is also identified to a single point. So, recall that X here is usually identified with X cross 1 in the mapping cone but in the cone itself, but when you go to mapping cone, this X cross 1 also is not a subspace, you will have to take some X cross half or some other T, any X cross T not equal to 0 or 1, those will be the copies of that, whereas Y is always a subspace of this mapping cone. The image of X cross 1 in CF itself may not be homomorphic text but there are other copies of X namely X cross T where T is not equal to 0 or 1. So, this is the modification, so do not make the confusion that X cross 1 is still a subspace of this mapping cone of the function F. So, there are various kind of constructions, the next construction is also very useful which is just doubling the cone. Cone is defined as the quotient of X cross i, wherein you have identified X cross 0 to a single point. This time what we will do, we will take two copies, so what we will do, we will take instead of X cross i, you take X cross minus 1 to plus 1, to begin with you have a double space here. Now, you identify both the end axis, X cross minus 1 as well as X cross 1 to two different points, single points. So, this is the picture, we have X here and X cross minus 1 to plus 1 like this, the cylinder, the X cross 1 goes to a single point here, X cross minus 1 comes to a single point. So, this is called the suspension of X. There is another terminology here, what are called as reduces surface suspensions which I am not going to define right now. That will be needed when you are deep into the homotopy theory right now, you do not need it. So, in the light of that, these are called un-reduced suspensions. For example, suppose X is a single point, then what is the suspension of X? It is no identification at all, you have to take X cross minus 1 plus 1 which is just minus 1 plus 1, homomorphic to minus 1 plus 1 right. So, the cone over single point was the interval, similarly the suspension over a single point is also an interval. More interesting things happen, just take two points, a space with two points and discreet space, discreet poly, not indiscreet poly, then what will be the suspension? Now, X cross I will consist of disjoint union of two copies of the interval right. One end point consists of minus 1 cross two points, the other one will be another plus 1 cross two points. These two things are identified to single points. So, what you get is like a diamond shape now, which is nothing but up to homomorphic which is circle right. So, the two point space can be thought of as S naught okay minus 1 plus 1 cross minus 1 plus 1, then identify the end points of these, then you get a circle. Actually, you better identify it as a diamond shape, but diamond shape is also homomorphic to circle okay. So, the suspension of S naught is S1. What happens to suspension of S1? Here is a picture, forget about the shaded part, this picture if you take just the boundary, there is a circle. When you take the suspension, what is this space? It is nothing but union of two copies of discs because the cone over circle is a disc identified along the common circle, the base. So, it will be nothing but the two sphere, suspension of a one sphere is a two sphere. Exactly same argument will tell you, suspension of any n sphere is n plus 1 sphere. When I say is n plus 1 sphere, I just mean homomorphic, that is all you can directly write down a neat homomorphism. If you use cost data science data and so on, it will become neat instead of the corners here, it will become round okay. There is a nice description of this suspension just like the cone at the function level. Namely suppose you start with a function from x to y and S of x, S of y are the suspensions, then there will be SF from Sx to Sy just defined by SF of a class Xt close to the class of Fx and t. To begin with, if you do not take class here, this is a map at the level of x cross minus 1 plus 1 to y cross minus 1 plus 1, which is just F cross identity. Whatever is identified in the domain, okay, all of them go to same points which get identified. Therefore, this map will factor down to give you a continuous function at the suspension level. The argument similar you have seen, cx to cy, cf, whenever x is a continuous function from x to y, it is similar to this one. So, you have suspension level and this suspension has all those typical properties. If you take x to x identity, S of identity is identity function of the suspension Sx and if you have x to y and y to z, another function g, then if you take g composite F and then S of that, namely suspension of that, that will be suspension of g, composite suspension of F. Because all those things are happening in the first coordinate here, g composite F is, okay, g composite F, that is all, okay. The t coordinates do not disturb the functoriality, the functional compositions on the first coordinate here, okay. Now, here are a few observations which I will leave to you to think about it. I will go through them slowly, to think about it.  Sx can be obtained by gluing two copies of cx, the course which you can write it as c plus x and c minus x along x, which is x cross y, identified. And a homeomorphism type is, the homeomorphism type is independent of gluing homeomorphism F from x, x. You can take any homeomorphism and identify c plus with c minus of x. The resulting space always homeomorphic presence. Construction of Sx is functorial property, which I already explained what is the meaning of this sentence. If x is a homeomorphism, then Sf will be also a homeomorphism. If g is the inverse, then Sf inverse will be Sg, okay. The last thing is, which also I have explained. Now, you have to write down some rigorous proof of this, that is all. The suspension of SN is one-dimension higher sphere, namely SN plus 1. To help this one, I am giving you the topological result here, namely, if we glue two copies of n-dimension discs along their boundary, okay. Via a homeomorphism, the resulting space is homeomorphic to n-dimensional sphere, okay. It is like gluing, simplest case n equal to 1, you take two intervals, close intervals, identify the boundaries by a homeomorphism from one to the other, namely a bijection, a goes to a prime, b goes to b prime, identify what you get to circle. So this generalizes to all the dimension dn, okay. So that is how S of SN becomes SN plus 1. If you do this one, then this is no, this is, there is no exercise here, this is not exercise. So this is like that, because you would have done this exercise already. And you would have also learned that the cone over the sphere, okay, is a disc of one-dimension higher. If you put them together, you will get this one, okay. So no exercise is to write down the details for each of them. Next, I will take one more example. This is called topological joint. This is actually a far more generalized than those suspension and so on. You can get suspension as a special case of this and you can see this, okay. So let X and Y be two non-empty topological spaces. If empty space in the product is empty and so on, so you do not want to get into that. We define the topological joint X star Y of X and Y to be the quotient space of X cross I cross Y, okay. So it is not just X cross I, it is not I cross Y, it is X cross I cross Y. If both X and Y are I, then this would be a cube, I cross I cross I, okay. That is the simplest picture you will have, all right. Now what I am going to identify each X1, 1 and Y, I mean when this T parameter is 1, it will be identified with X2, 1, Y. The third coordinate must be the same. Then first coordinate any arbitrary X, they are all identified as single point. For each Y, this is called. So at one end, I am killing the identifying the X to a single point for each Y, okay. You can write X cross Y cross 1, right. Instead of X cross 1 cross Y. In that each for each Y, the X is identified single point. So that is one thing. Same thing I want to do the other way around. At 0, for each fixed X, I identify Y1 to Y2 for all Y1 and Y2. X is the same. So for every X in X and Y1, Y2 and Y, okay. I started non-empty spaces. If Y is empty, then you have to define what is X cross Y. So if you take product, then it will be empty. So you do not do that. You would like to have X star Y as X itself, okay. Okay. Maybe you would like to define it as empty. That may be better. So I am not very sure of this one. Okay. If you need some change, you can define it like this, okay. Inputively, what is this topological joint? X cross I cross Y. It has union of lines parameterized by X comma Y. This is just line segment. Union of lines parameterized by X comma Y. Okay. But when you identify all the X to a single point, what you have is from a single Y to that space, you have a line segment. Only for Y2 do we have a line segment and so on. So I will let you know X1 with a better example. Okay. Here this is not an example. We are trying to explain this in the abstract sense only. So what it is, is look at the class some X, some Y. Okay. So T is never identified to anything. So as T runs from 0 to 1, you will get a line. This line you can think of as starting at X and ending at Y. In other words, you see in abstract sense it is creating lines from X to Y. When the cone what it has created, an extra point was taken and there was lines from every point of X to that extra point. Right. That is the cone. Here what I have done, here we do, instead of just one extra point, a space is already given. X is given, another space is given. Now for each point of Y, you draw a line from X to Y. Where do you draw, that is the point. So it is not happening in any RN or Euclidean space or any vector space. It is abstractly happening. Okay. That is why you take X cross I cross Y and do the identification. Okay. So to justify this explanation, let me give you a simple example. There is a typo here. Start with X to be a line segment PQ and Y to be another line segment UV. Okay. Denote two line segments in R3. X equal to PQ this is. Okay. Which are skewed. The two segments must be not intersecting non-parallel. That is called a skew lines. So how to ensure that? There are many ways of ensuring it. Namely, you can look at fix one point as origin. Then look at Q minus P, U minus P and V minus P. So you get three vectors. These three vectors must be independent. So that will ensure that P minus Q and U minus V are not parallel vectors. So line segments are not parallel. Neither they intersect because if they intersect, there will be only two vectors here. Okay. The third vector will be already one of the independent either 0 or one of the difference of these two. Some of these two. That is all. Okay. So the topological join of this X and Y, the two line segments which are skewed is nothing but the convex hull of this. Take the convex hull of all these four points. Namely, what is called a tetrahedral. The convex hull of three independent points would have been a triangle. Okay. So there is a fourth point which is not in that plane. So what you get is a tetrahedral. Which is same thing as take arbitrary point in PQ from P to Q. Join it to some arbitrary point in U and V. Okay. Between U and V. Take the collection of all these. So that will be the join of X and Y. It is also equal to the just the convex hull of PQ, U, V, which is tetrahedral. So in some sense the join is generalizing even the construction of convex hulls. Okay. Now we can do some elementary algebra of sets on this star. The star is joint X star Y. X star Y contains both X and Y as subspaces. Only at one point the points of X are identified at namely when the T coordinate is one. Similarly, when T coordinate is 0, points of Y are identified. If you take T coordinate not equal to 0, the whole X cross Y will be there. So both X and Y are subspaces. In fact, to take X going to X, 0, Y, there is no identification of X here. The Y coordinates gets identified. So this is a nice copy of X on the 0th phase. Similarly, Y going to any X comma 1 comma Y would be a inclusion of Y. Okay. X comma 1 comma Y. These are subspaces of X star Y. Okay. Just similar to the phase of the cone or suspension and so on. The topological joint is both commutative and associative. What do you mean of this? Up to homeomorphism, X star Y is same thing as Y star X. So this follows by the commutativity of the product. X cross Y is homeomorphism to Y cross X. Okay. Similarly, it is associative. X star Y bracket star Z is same thing as X star Y star Z. All that you have to do is first have this law in the product space. X cross I cross Y cross I cross Z. You put the bracket wherever you like. So when you take quotient space, that is the X star Y star Z or X star Y star Z. But the identifications are exactly same. Exactly same whether you do it this way or this way. Therefore these two spaces are homeomorph. Okay. Similarly, this is easier. This is commutative. Okay. Suppose X is a singleton space, then what I have? I have X cross I cross a single point. This is X cross I. And then I make identification. Okay. So what is this? At 0, I am defining all the X to be single point. Right? So that will become like a cone. When I come to 1, there is already only one point there. Right? Why? Why there is no identification. All the points of Y should be identified but there is no identification. Therefore, this is just the cone. Okay. So this is one of the reasons why we, even if a single point, it is not X. The operation is not X, it is CX. So if you want an operation to be, to have identity, then you have to take Y as empty set. That is one of the reasons why I have taken. If Y is empty, then you define X star empty space as X. This is just definition. It does not follow logically because if you take the product space with an empty set, it is empty already. Okay. So what we know is if you take one of the things as single point, then this is the cone. If you take one of the two points, then what you get is S naught of S naught star X by definition. These are two points. This will be exactly same as the suspension. Okay. So with the kind of explanation I have given, it should be clear that why this is true. S naught cross I cross X will consist of two copies of X cross I. When you perform identifications around X, we will get two cones and they will get identified because of identifications for S naught. Okay. So you get S naught cross star X is S X. The special cases of this I have already told you. Once it is S X, S of S naught. So S naught star S naught. We discussed this. What is this? It is S of S naught which S 1 and S naught of S 1 which is nothing but S of S 1 which is S 2 and so on. Right? So S naught of S naught, S naught star of S naught is homomorph of S 1. Inductively, we can show that S n star S m is homomorphic to S of n plus m plus 1. All right? You can write S n star S m as inductively S naught star S n minus 1 here star with S m. Okay. By associativity, you take the first this star that will be equal to by by induction star S of n plus m. But now I am taking S naught of that. By this S naught of that will be one more dimension higher namely S n plus n plus 1. The induction star said n equal to m equal to 0. Okay. So that is a proof. Okay. So we can stop here.


I've always said that I imagine this convening as a, in the style of a family reunion. You know, when we come together to honor a particular thread of heritage, a particular thread of tradition, because we have a connection to one of our many surnames, and we're gathering around that right there. And when we go to a family gathering, one of the things that, and especially a family reunion, one of the great gifts is we get to meet people we didn't know we were connected to. Today, this institute's imposition is a manifestation of an intention, of an idea that was spoken, that was conceived and spoke in the loud of Anisa Portres and Lou Moreno back in 2013 in Boston, to which Brian Herrera committed himself, and along with Ben Garcia Romero, Vidalia Pumus, Georgina Escobar, and Abigail Vega, has led the way from taking this idea, this manifestation, this presence, all of us here together. And it's quite astonishing. As I really start to think about how we treat our visionaries, how we treat our visionaries, I read this little line, she's a visionary, and I really train other visionaries, and I really train other women or visionaries, and I really do feel like if we're going to talk about family, I really do think that we need to look to those visionaries and take care of them, and produce their work, and make sure that they're okay. Like, I always think about her voice when I teach, and she had a tiny little human-accented voice that was just as small as it was, it was fierce, so I think about all that energy that she brought with her into the teaching, into the classroom. I never studied with her, but I certainly was influenced by her work, and I think at that point in New York, she was certainly one of the most influential playwrights. I wish she still were, but she's a remarkable person in American theater. The Fortnance Institute's mission is to celebrate, to amplify, to elevate, to archive, to preserve, to make more visible the living legacy of Irene Fortnance. What is your impulse coming out of today? Is there anything you'd like our witness, our support, and our validation as you carry your impulse forward in your own practice and in your own communities?


Databases, a database seminar series at Carnegie Mellon University is recorded in front of a live studio audience. Funding for this program is made possible by OttoTune. Hi guys, welcome to another data seminar series. We're excited to have Ben Johnson from flyup.io. He is the author of Lightstream, which he's going to talk about today. Prior to this, Ben also built another popular database called Bolt, which is written in Go, a key value store, which has email he's discontinued, has taken off on his own. So we're super happy to have him give a talk today about Lightstream and SQLite. If you have any questions for Ben as he gives the talk, please unmute yourself and say who you are and ask your question anytime. We want this to be a conversation for him and not talking to Zoom by himself for an hour. So Ben, thank you so much for being here. The four is yours. Go for it. Awesome. Yeah, thank you so much for putting this on. These talks are great. So I really appreciate coming on here. So yeah, again, my name is Ben Johnson at Fly.io. I wrote a tool called Lightstream a couple years ago, and it's centered around SQLite and making that available in a lot more use cases. Specifically, I want to make this idea of a single node deployment, which I started maybe 20 years ago in the industry, and back then running on a single server was not a crazy thing. Whereas these days, it seems like everybody has 20 nodes in their deployment for a blog. So trying to make this, you know, this simplified single node deployment kind of cool again. That's my hope. So if you've watched any of these other talks, you know, they're fascinating and there's a lot of really awesome cool stuff that people present on, you know, a lot of stuff like high performance, you know, people are sending millions of queries per second through databases, people talk about scalability, where you might have just something, you know, a thousand node cluster and doing whatnot, or people do fancy data structures like CRUTs and whatnot. And this talk is going to have absolutely none of that, unfortunately. But there are some other interesting things and the industry that I think are going on that are kind of cool to look at that aren't centered around this stuff. So hope it gives another perspective. So first of all, Lightstream, what is it? It's a streaming replication for SQLite. So it'll take your database in SQLite and continuously back that up to either Amazon S3 or, you know, Azure has a durable storage that is Google.  You're basically just trying to continuously stream out your SQLite data to another location for durability. And this kind of, this high durability, high durability really gives you the ability to safely run a single node and really minimize the window of data loss you can have with it, even in the worst case scenario where it, you know, catastrophically fails. If you're not familiar with SQLite, it's a great little tool. Its motto is small, fast, reliable, choose any three. And it's an embedded SQL library. It's a pretty full featured SQL library. It's got CTEs. It's got, you can do all kinds of joins. And there's a ton of stuff you can do in it, honestly, for such a small little library. And the databases are really just these individual files on disk. So they're pretty simple to reason about. If you want to send someone the database, you don't have to do any kind of special fancy commands, you can just send them the file and they have the database.  You have to go over to another machine and talk to it. And with SQLite, there's none of that network communication. You don't have even IPC, really, except for some locks. You can do multiprocess, but generally it's usually a single process. And it's really just, you know, read, write calls to a file system, essentially. So as far as the kind of the small part of SQLite, it's about 130,000 lines of code for the core database. There's other things you can add on extensions that are actually typically built in, like a full text search and some JSON support you can add on there. And, you know, that core database really can do quite a bit. It's all the transaction layers, it's paging data in and out. You know, it's pretty fascinating what they've done. And you compare this to something like Postgres, which is millions of lines of code, you know, it's just, it has a ton more features, some are more esoteric, some are just, you know, it's more robust. But, you know, for SQLite gives you a lot in that little package. And from that small package, you know, they've really, in addition to having all the features that, you know, a lot of applications use, they have this massive test suite. And this is kind of the reliable part. So it's about a million lines of code for this test suite. And all those tests are even parameterized. So each one's running kind of multiple tests within it. I think it takes days to run when they do a release. So it's pretty extensive. And finally, you know, SQLite is considered the most widely deployed database in the world. So it runs on your phone, it runs on your laptop, it runs in appliances, it runs on airplanes. It's, if you have even like, it runs on all kinds of weird obscure CPUs. There's one VFX works I never used, but apparently runs on it. You can run on ARM, you can run on AMD or on x86, you know, the sky's the limit really. It's really a great tool to have around if you need a database. So before we get to the actual light stream part, it's really good to understand how SQLite is structured internally. And it has a really simple layout. So this shouldn't take too long. The actual physical layout, again, is just a file on disk. And the file is separated out into fixed size pages by default. It's about four kilobytes. You can change that down to 512 bytes or to 64 kilobytes. And, you know, each of these pages are just added in their sequentially. So page one, page two, page three. But then logically, they're actually structured as a B plus tree. So B plus tree, if you remember, all the actual data is stored in the leaf nodes, so the ones at the bottom. So you write, you know, pages two and five would have your leaf actual data. And then the structure above it is actually, you know, nodes pointing down into kind of a sorted list. It's kind of like a sort of tree and how it's structured. And those branches just kind of point down to the leaf nodes. So this kind of layout works great for, you know, read optimized data, which SQLite is used used a lot for. If you have something like an LSM tree familiar with those like level DB, that's more right optimized layout. So yeah, really simple layout. Lightstream is a physical replication layer. So we really just care about pages. We're actually don't even care about your underlying, you know, your rows and your schemas and stuff like that. We're actually just copying out pages. So also another important thing to kind of understand about SQLite and, you know, where it came from in its use cases and why those are kind of evolving. It's good to understand some history. So SQLite originally started in 2000. And for the first 10 years, they just had a transaction mechanism called the rollback journal. And it works pretty simple. If you want to make a change in the database, you know, transactional name, you would actually copy out the old version of a page to a file called the rollback journal. And then you can make changes inside the database. This is really simple. You know, if anything fails, you can always copy back from the rollback journal into the main database and you get your state from before the transaction. Now the problem with this is you get really bad concurrency because, you know, if you have a writer that's kind of mucking around in the pages, you can't have other readers reading those pages at the same time. So you have either a single writer or multiple readers. So this really limits it to kind of the single user software. If you think of like the messages app on macOS actually stores data in SQLite locally. And, you know, that's really a single user. You just, you're the only person using it on your laptop at a time. So you don't really need that concurrency. That's a good example. Or like phones, for example, it's just you using it. Now what I would consider kind of modern SQLite is maybe in the last 10 years. The write ahead log was added to SQLite. And this just massively increased the reconcurrency. So the way the write ahead log works. And please, again, stop me if you guys have any questions. If you guys, if you have a, it basically works the opposite of a rollback journal. So instead of copying the old version of the page to another file, you're actually going to write the new version of the page to another file called the write ahead log. And it's an append only log. So every time you make a change to a page, you're going to add on, you're going to kind of tack it on the end of that write ahead log. So this gives you better reconcurrency because you're able to write. And at the same time, you're going to be able to have readers reading old versions of those pages because multiple versions exist. You get better performance from being able to do this append only.  And each reader gets its own snapshot at the point in time when it started. So it's, it's super flexible and super easy to kind of use it. It's serializable isolation. It should be familiar with isolation levels. If you're looking at more like distributed consistency models, this is really what would be considered more of a strict serializability where you have a total order at the database as a whole. And you also have kind of real-time constraints about when data, you know, when you are able to see transactions when they're applied from the readers. So, you know, all in all this, you know, even though it's just a little library that's running locally, this can handle a ton of concurrency. I can usually handle a kind of small to mid-size web applications out there that might be experiencing kind of in the hundreds of requests per second. But just great for those, which is honestly probably like 9% of applications out there. Like for all of the Facebooks in the world, there are so many more people that have an app that gets, you know, 20 hits an hour or something like that. So that's, that's kind of more of that where we're focusing on with this. Finally, the last thing is that, you know, in addition to the improvements in SQLite itself with its concurrency, the cost of computing has just like plummeted over the last 20 years. I tried to pull a little bit of data on here. So back in 2000, you know, you might have a server 32 bit CPU with maybe a gigahertz, you know, might be 700 megahertz processor in there, single core. We're not even talking multi-core. You know, might have a gigabyte or half a gigabyte of RAM and you're running kind of spinning disks back then. So those are getting maybe 75 megabytes per second. And you're probably paying 10 grand for that server. Like it's really expensive. So, you know, kind of the idea with NTR architecture is that you get this really beefy, optimized database and put it on the most expensive server you can afford. And then you have a bunch of smaller, cheaper application servers that can offload some of that like HTML rendering and whatnot, some of the business logic off of the database server. But, you know, over the course of 20 years, we now are just pulled up, you know, Dell's website and you can get kind of their baseline seems to be like a six core, three gigahertz 64 bit processor. You know, you're getting gigabytes and gigabytes of RAM. If you want to go to AWS, you can find machines that have a terabyte of RAM on them, which is just mind blowing, honestly. And then finally, you have hard disks that are SSDs, NVMEs and these get gigabytes per second of throughput. So, you know, we're just in this completely different world from where we were just two decades ago. And the server only cost you maybe a thousand or two. It's relatively cheap. And then finally, the other alternative that's really popped up of the last, I don't know, five, 10 years are virtual private servers, where you can get a machine that's beefier than what you got for $10,000 back in 2000. And you can just rent it for five bucks a month, you know, digital ocean, you know, I work at fly, we do that kind of stuff as well. So it's just, it's amazing how much computing power we have now. And, you know, what we can do with it and what that looks like and how it affects our tools. And, you know, now we have faster computers, the database isn't necessarily like a bottleneck for performance. So that's not always the most important thing as cool as it is to have like a million transactions per second, like most applications don't need that. And really, actually, we're starting to get to this point where a limiting factor is really the speed of light. So instead of trying to get your machine faster, your code faster, you just need to get your data closer to your users. Or if you want to process the data, you want the data closer to your application, not in another database on another machine. For example, like if you have like a Postgres server, you want to connect to from your application. And if you're running even within the same region on AWS, you know, you can have up to millisecond of latency just for a query, that's not even the query processing time, it's just simply getting into the database server and back. Whereas if you looked at like SQLite, we really don't have any layers in between our application and the data, so the overhead is about 10 microseconds per query. So, you know, there's just orders of magnitude difference. And if you're looking like a point query, looking up by a primary key, or if you're doing a simple range query, a lot of the actual request processing time with that, you know, might be in the tens of microseconds to actually just do those lookups. So you're, it's amazing how much time we have just to go reach the database and come back.  And, you know, the manuals for like Postgres or for Oracle are thick, like they're massively thick. There's a lot to know about those. They're really complicated. A lot of times you have to set up users and roles and grants and you got to open up ports and, you know, there's a lot of stuff that are just kind of a pain. So if you can remove all that and just have a simple storage primitive like SQLite, that can be great. And the last or another thing is the speed of development. With something like Postgres, you have issues called n plus one queries, where one query might get 20 rows back, but then you know, I needed to make a query for each one of those rows back to the database and that can really be a performance hit. And the last thing people want is safety.  If you have like a fire in your data center, for example, and that's kind of where SQLite falls over is safety, not that it doesn't store data correctly and not corrupt it. But if your machine dies, it's a single no database. And that's kind of where light sharing comes in. It's meant to be kind of this this missing piece as to why a lot of people can't use SQLite for their applications. It's just not safe enough. Okay. Any questions so far? I'll take a quick drink. Talking too fast. What's that? Sorry, did you say something wrong? I think Andy said, I think we're good, which is what I was saying. Yep. Okay, cool. So, okay, let's jump into actual Lightstream and how it works and what it does. So SQLite's wall, the right ahead log. That's the only mode that Lightstream will work with. And once you see a little bit about this, you'll understand why. So SQLite's wall journaling mode basically writes those new pages in a transaction to a separate file. So if your database is called DB, then your wall file will be DB-wall. And for example, here our database file, we have four plus pages inside of it. Excuse me. And right now it's just the database file. So if we start our transaction, then we're just going to be reading database pages, share from the database file. If we create a transaction and do a write and commit it, we might update pages one and three. So now a transaction that started before this write would read just from the database. But if it started right after this transaction, it'll read pages one and three from the wall, and then pages two and four from the database file. So it's able to get its own snapshot, the version of the database for that point in time when the read transaction started. And then we add another write transaction that updates page two. It's going to read from, you know, if it started after that transaction, it's going to read page two from the wall. If it started before that transaction, it'll read page two from the database. So again, each transaction is getting some snapshot. And then finally another transaction can even update pages that are already in the wall. And depending on when you start your read transactions, those will read from any pages kind of before that read transaction started. So that's the basics of the wall. One issue you might notice here is that the wall just keeps growing. And that's a problem. I mean, disk is cheap, but it's not that cheap. So we actually have to do this process called checkpointing or SQLite will do it. It's called checkpointing, where the last version of every page will get written back into the database. We need to wait until all reads or all transactions are, it'll basically block new transactions and wait until any existing ones are done. And I'll do this copy real quick and then you can resume processing. So a lot of SQLite, it's usually best to have very short lived transactions and SQLite. It doesn't work very well if you have transactions open for a long time. So essentially here, we'll have, you know, page one goes back to page one database, page two goes to page two. And then this version, page three goes to page three. And once all this is in the database, we can then truncate the wall and it'll start over. I think we have a question from Katie. Do you want to unmute us? Yes. Hi. I am Krithi. I am a master student at a university of Waterloo. So the wall is we are, every transaction is creating a paid snapshot and adding it to the wall. My question is how do scans work with this generating strategy? Sure. So scans are going to, so you're going to be, if you scan the data, if you want to go, you're basically looking at, actually you can think about it without the wall first. If you look at just the database, we have, you know, logically it's a B tree structure. So if you wanted to scan, you might start from, if you're scanning say a primary key, you can seek to the primary key by kind of traversing the B tree. And then you'll start from that place and you can scan along forward to the pages, the leaf pages in the B tree to actually to read that scan. The only thing the transactions at the wall do are you can update versions of those individual pages so that new transactions, instead of reading from the database, they'll just read from whatever is in the wall. So if you updated a name in a row, that might only update one page with an, you know, maybe a leaf page in your, your B tree and your transactions just jumping to the page in the wall instead of reading from the database. Thank you so much. Okay, good question. Any other questions? All right, cool. Yeah, thanks for calling it out. Well, so that's checkpoint. We're basically copying the last version back into the database and we can delete the wall and it'll start back over from the beginning. And we just keep doing this over and over again, and that's how we can manage the size of the wall by default. The wall gets up to about four megabytes, so it's not huge by any means. You can, you can adjust that, but that's kind of where it starts to try to push the data back in. So checkpointing. So we're kind of getting these new versions of the wall every time we start writing out. Now what Lightstream will do is in the current version, it checks every second for any changes to the wall. And if anything is updated. So if you do a write to the wall and maybe another write to a wall within that second, then Lightstream will check in and say, oh, hey, there's two new transactions. I will copy those out to what it calls a shadow wall, which is basically just the directory to the side of the database. And then another transaction comes in on the wall. Lightstream can detect that and then copy it over. And the wall has some check summing within it. So we can actually verify consistency within that. And yeah, essentially we just keep copying to the shadow wall. And if we actually get a checkpoint on the wall and truncates, we can detect that and it'll start another transaction and we can copy that over to the shadow wall. So we're basically kind of replicating every version of the wall in the shadow wall. And the shadow wall just acts as kind of a staging area for us to later push out to durable storage. So the shadow wall, we're going to have essentially all these versions of the wall from SQLite. And we're going to start just basically pushing right over to S3 or whatever storage you want to have. And once they're safely there, we can just delete them from the shadow wall. And now they're just, the shadow wall is really not meant to store a lot of data very long, just very temporary. And we really have kind of two concerns that's separating here, where we're trying to move stuff off of SQLite as quickly as possible, because we don't want to have the wall grow. We have a lot of transactions going on. But then, so we're trying to get that as fast as possible. But then on the S3 side, we might not want to push up as often because there's a cost every time we push up to S3. So we can, we can change that if we do it every second, if we do it every 10 seconds, just depending on what the end user actually needs and how much they want to spend on it. Hi, so I have some questions maybe related to that. I think you just answered the first, which is how periodic is periodic? It seems that this is up to the end user. Yeah, by default, it originally started as 10 seconds, but I actually dropped it down to one second. Okay. And go ahead. Yeah. Okay. And for a second question is, when you're doing the copying to S3, are you batching things together? Are you copying it as is? So when it's copying up to S3, it's doing it kind of as wall segments. So we're actually, every second when we copy out from the wall inside of SQLite, we're taking like a chunk, like a subset of the wall. And then we push up to S3, or we're basically copying those chunks up. So it's not so much of a batch, we'll kind of concatenate some together. But yeah, we don't batch a whole lot. Thank you. Good question. So then another thing that Lightstream does is by default, every day, it'll do a snapshot of the whole database and push that up to S3 separate from the actual wall file. So that means once you have a snapshot, you just simply need to pull that down and then replay every wall file that occurred after that. And then you'll apply those snapshots, basically do a checkpointing process for each wall back into the snapshot. And then you'll get the result will be a byte for byte exact copy of your database. And that point in time where that you're replaying up to. So, you know, given that you have a one second delay going up to S3, you really only have a window of data loss of one second, which is pretty good for a lot of applications. It's not quite synchronous, but it's close. Yep. Or I could read it out. Sorry, I was just with the wall fragments get to S3, are pages entirely contained within those? Or could you have a page that split across two of those segments? Sure. So the segments that go up, it's going to be contiguous transactions. So it's going to be full transactions. So you'll always have, you'll never have a transaction that's split across segments. And all the segments, all the pages are within those transactions. So you'll never have a partial page go up within transactions. It's always full pages and full transactions. You might have multiple transactions within a segment. Yep. Good question. So, you know, long and short is that, you know, the stuff that Lightstream's doing, it's nothing super complicated. A lot of it's just like copying over chunks and moving up chunks and then pulling chunks down later. And, you know, the goal was really around simplicity. Like, you know, if something goes wrong, it's pretty easy to figure out what happened and we can debug things. You know, there's always more complicated things we could probably do. But this actually seems to work for, you know, a lot of use cases out there. People seem to have taken to it. Any question from Kriki? So I might be making some assumptions, which may be wrong. But my question is that the logical tree, it contains pointers or page addresses, right? And so when these pages are moved to S3 and are used as backup, how is the SQL light database restored from this backup? Sure. Yeah. So the page pointers that are within the B tree, that is right. They are pointers with their pointers to a page number, not necessarily a specific version of a page. So the actual process, when you actually do restore from S3, you're copying these pages back into the same position. So if you backed up pages, you know, one in three for a transaction, when you bring it down and you checkpoint it back into the original database, you're copying those pages back into where pages one and three would be in the database. In the database file, you can, you know, it's positional. So since they're all fixed size pages, if you wanted to find page three, it's going to just be, you know, the page size of four kilobytes times three, and that'll give you the position of page three. I don't know if that makes sense at all. I can try to explain it further. It may, I do get it, but I have questions, but you can move forward for now. Yeah, go for it. So I continue to last piece too with Lightstream. A lot of these, you know, all this data will start to build up over time. It's nice. One thing about Lightstream is you can do point time restores for any data you have. So if you want to, you know, if you drop the database on accident, you can go or someone corrupted data in the database, you know, two days ago, you could go back and actually restore up to the point just before that corruption. But you don't want to hold your data forever. So there's a retention enforcement where by default, it's every day, it'll delete out stuff that's a day old. It'll always have at least one snapshot in there before it deletes or after it deletes. So it should be safe, but you can keep, you know, for example, a lot of people will run maybe seven days of retention and you do a snapshot every day. And the idea with the snapshots is that the more often you do snapshots is the fewer number of wall segments you need to replay afterward. And it speeds up your restore time. So if you have a lot of writes and a lot of wall files, you know, it could take seconds or minutes to restore all that because I asked to download all these little files from S3 and replay them all. So yeah, more snapshots is just fewer wall files. Yeah, so that's kind of long and short of how Lightstream works. I can jump in if you have questions. And, you know, a big piece of Lightstream and honestly, most databases, even with software is just how it fails and making sure it fails well and what those values look like. So, you know, I mentioned this before, Lightstream is an asynchronous replication tool. So that means that there's no acknowledgement that anything's been replicated to the client before the client receives an acknowledgement. So that means you could have a data loss window if it can't communicate with S3 or, you know, just that one second of delay between pushing out. So in a catastrophic failure, you can lose up to, you know, X number of seconds depending on what your sync interval is. But if you are doing like a deploy, if you're trying to shut down, it does try to make a best effort to try to push out any outstanding data before it shuts down. On the restore side, we're not doing really any optimizations. It's meant to be kind of a dead simple tool where the wall is really just pushed up as is. And even if you have kind of duplicate wall pages in there or, you know, pages in the database, it doesn't try to merge those or compact those. And again, I mentioned this before, but, you know, more wall segments does mean a longer restore time. And, you know, we're doing compression up there. LZ4 is what we're doing right now. It seems to work pretty well. And we're doing, I think, on the lowest compression setting just to get the performance out of it, as far as a trade-off of compression size. And we're getting to some of the costs as well that kind of pushes towards that. Yeah, I think I cover all of this stuff. Yeah, snapshots improve restore performance. One quick comment or question from Jared. Is that how I put it? Yeah, yeah, sorry. I can unmute. I just got a little louder if I remember around me. Ben, just maybe for the benefit of others who are thinking about these things as well, how would you compare the use of virtual memory and memory map files with the wall mode compared to the rollback journal and how those modes operate differently? And the reason I ask this is, of course, like, if you're trying to map this to the browser or other places, since SQLite is so portable, you have places where it does not fit into the VFS architecture, right? Sure, I mean, I map stuff. I can't remember exactly. I know there's some things you can turn on and off. The rollback journal and the wall can be separate from that. That tends to be kind of its own concern as far as the journaling mode. I know that I haven't done a whole lot of stuff as far as WASM and the browser. I mean, I've seen a lot of people do some cool stuff on SQLite and what you can run in the browser. I think there's really some interesting stuff of people running WASM on the edge. And Lightstream itself doesn't lend itself well to that. Actually, the next slide will actually tell you why it doesn't lend itself to running in the browser. There's some stuff we're working on in the future that we are hoping does work well We do want to actually run this on the edge for people that are running WASMs, if you have like Dino, I don't know if you guys have heard of Dino or some of these other like Bercel, for example, they run a lot of code on the edge, kind of JavaScript code on the edge. And we think there's a place for SQLite there too, even though it doesn't seem like it naturally fits. I don't know if that could answer your question, Jared. No, no, that hit the exact, like literally down to like those people, like the exact problem that we're like, yeah, like, like think about Lightstream and other tools from. So yeah, that's great. Yeah, yeah, cool. So another failure issue with Lightstream is that you can't run multiple instances of Lightstream all pointing to the same destination, which might seem very obvious, but there's a caveat that people don't think about with this. So, you know, it's uploading snapshots and it's doing its wall segments and it's doing this retention policy enforcement. So if you have two instances trying to replicate to the same place, they kind of run over each other and start deleting each other's data. And obviously, you shouldn't run two instances at the same time, but kind of modern deploys have this thing where they will do rolling deploys. A lot of times where you might have two instances of your application running at the same time for just, you know, maybe a few seconds, and that can kind of corrupt your database or your backups. So certain platforms, this doesn't run on like GCR, Google Run or Google Cloud Run. Some of those actually fly, we have an ephemeral version that this doesn't work very well on. So you need to have like a persistent storage along with your instance to make sure that there's only one instance at a time, typically. So that's one caveat that people kind of run into. And then another issue too is that restarting your instance will incur downtime.  And then you deploy new ones and they kind of like create new nodes and the old ones just disappear and like it's very ephemeral stuff comes and goes, you don't have a single server you're deploying to. So Lightstream actually doesn't work great for ephemeral stuff right now. It's really built for kind of an older VPS model where like DigitalOcean, for example, you spin up a VPS, you run it on there for a bit.  So that's kind of a caveat with Lightstream. You know, one thing that drives Lightstream and one thing that makes it kind of interesting is the cost model. When people hear continuous streaming back up to S3, that sounds really expensive, but it's shockingly really cheap. And that's kind of the whole reason this thing exists.  So they, they don't charge you for the ingress fee to actually upload the data as far as the bandwidth, that's all free.  They do charge for put requests. So every wall segment we push up is going to cost us, you know, half a cent per thousand requests, which is super cheap. And if we're replicating every second and we actually have writes just constantly for the entire month, the maximum you'll pay is $13 a month. So that's really at the high end. You can drop that down to every 10 seconds. So we have about $30 a month to do backups. But really in practice, like people aren't getting writes constantly every second. So Michael Lynch, a while back, he actually wrote up his experience with Lightstream. He pays three cents a month for his web app to do backups. You know, it's great. It's, it's pretty cheap. So it's a nice alternative for a lot of apps that don't need, you know, full synchronous replication and, you know, don't want to pay a lot of money, especially with a lot of like free, free tiers on a lot of these systems we have want to fly, but like a lot of these places do, we can spend stuff up super cheap and run it there. So that's a great alternative. So a lot of stuff. So that's all with Lightstream, kind of how Lightstream works overall is fairly straightforward, I think, I hope. It's a physical replication, but we've learned kind of a lot of lessons from that stuff with Lightstream. And we have, we're doing some future work, not on Lightstream necessarily. So Lightstream runs a separate process. And that that's great because it's kind of an operational concern. You don't have to build it into your application. You can run legacy applications without them having to change it all. They just run on Lightstream and Lightstream ships off the data automatically. But the problem is, you know, with all that is, it's a separate process and it really just doesn't have a lot of fine control about when transactions happen and, and what it can do. So, you know, one big feature that are probably two big features that people wanted from Lightstream is to be able to have re-replication. So they want to be able to replicate out their data to different parts of the world or to different servers. So they have, you know, high availability and redundancy. And that second feature is high availability. They want to be able to do deploys without having downtime, which is, you know, pretty critical for a lot of apps. So we started with a thing called LightFS, Light File System. And that's, that's a lot of what I work on these days. So this borrows a lot of ideas from, I'm not trying to make two presentations. It really borrows a lot of ideas from Lightstream or just kind of repackages of them in a way that we can make it so we can distribute SQLite databases in a kind of natural way. So it's a fuse-based file system. So this is a user, user-space file system. It's not compiled into the kernel. You can create them on your own. There's all kinds of used file systems out there you can use. They're kind of fun to make actually if you ever want to download and try one. And essentially we want a cluster of nodes that all share the same SQLite database or databases. And, you know, we still borrow this restriction of a single writer at a time because that's kind of how SQLite works. It doesn't have any concept of overlapping transactions or optimistic locking or anything like that. So we do have a single writer elected via a console lease. And the nice thing that, you know, some of the features we've added with LightFS is that we are able to do these kind of point-in-time restores you can do very quickly. Yeah. So there's some kind of interesting stuff that we've pulled out of Lightstream and made into this new system. Yeah. So I'll touch on a couple of these points here. You know, people wonder why fuse specifically. SQLite does have a virtual file system built into it, but it really requires people to actually load up the extension. And it's just kind of one extra step for people to do.  And, you know, we're just removing into this world where developer experience has really kind of become a bigger thing, you know, even more important a lot of times in performance or even sometimes cost honestly. So being fused, you know, fuses this file system layer. So that's really how SQLite communicates us through the file system. So we're just kind of injecting ourselves into there. And we can get a lot of fine grain control around, you know, if, you know, someone tries to write on a replica, we can prevent that, you know, we can redirect rights, we can do all kinds of kind of fancy stuff. And it also gives us stuff like multiprocess support. So we're at the file system layer and set up in the process. And, you know, this thing needs to run as a single API server as well. And if you had a multiple multiprocess system on a machine, you can't really easily do that. So that's why we're using the fuse file system. People also wonder why we even bother clustering SQLite. That seems kind of crazy. Why not just use Postgres? Which is, you know, again, I don't want to not Postgres, I think Postgres is awesome. But it's very heavy compared to something like SQLite. And, you know, we want to cluster it because we want to increase availability. We want to do these deploys at that downtime. And we really want to be able to replicate to the edge, not just like that's possible, but that it's dead simple. You know, we don't want people to think like, oh, now I got to read up on all this stuff and set up these things. We basically want to have people just kind of have a switch. And more or less, it just kind of works. I mean, there's a few caveats there, but it should be relatively simple. And we think we could do that. And the reason to replicate to the edge is that, you know, a target for a lot of people with web apps is to do 100 millisecond request time. That's kind of the amount of time it takes. If you want to, you have to feel snappy and like real time, then 100 milliseconds is kind of a human perception of what feels snappy. And for most of us, you know, people that are over in CMU, you guys are right by the US East One region for AWS. I'm sure a lot of websites feel really snappy to you guys, but people around the world have a much different experience. And just kind of, in my head, I always think of kind of these rules of thumb, like going from the US to Europe is about 100 milliseconds of latency. You know, that's not even with any query processing or request processing. That's just literally the speed of light going from the US to Europe. And then if you go to US to Asia, you're looking at a quarter of a second, and you know, things just start to feel very sluggish, especially once you add on additional request time or request processing. So if you can, you know, push out servers and make it cheap for people to run through data in a lot of places, then I think the web will just feel a lot better for a lot more people in the world. So yeah, point of the edge sucks, and we just want to improve that. Another, you know, piece that people are curious about is why we chose to do leases with console. You know, there's a lot of different ways you can do this. There's raft and it's been really popular in the last 10 years. You know, we want to make it really simple. And like I've written a raft library at CD, the original raft library they had was written, I wrote it. It wasn't great. So I'm not bragging that actually took it out. But I like I've had experience with raft. So it's a great protocol if you really need that really strict distributed consensus guarantees. But there are a lot of applications that can use looser guarantees and that and really people want operational simplicity. And that's the one that's kind of loose membership into this ephemeral world or we have Kubernetes and things spinning up and those all the time. So, you know, we use, I think it's somewhat novel way to do this where we do have this a transaction ID that's monotonically monotonically incrementing for the most part. And with each transaction ID we associate with the checksum of the data and the database. So the actual full state of the database to check some on every transaction, which sounds expensive, but we actually do this in a very incremental way where when a page comes in, if you get a new page, we're going to do a CRC 64 of the page number and its data. And then we XOR that into the checksum. So then when we do an update to the page, we kind of take out the old checksum for that page and add in a new one. So we always have this rolling checksum. And actually thanks to Andy actually didn't know the name of what this would be called actually had to ping him and ask him. So we have this rolling checksum that always gives us the exact state of the database. So we can do this constant integrity tech and checking between all the nodes. So, you know, there are some failure cases where a, you know, with asynchronous replication where you can have nodes that you might have a primary that gets a transaction, but hasn't pushed out to record those cats, you know, it can fail, lose leadership and another node comes up and now you have this divergence between nodes. And this checksum really ensures that every node, you know, rejoins onto another node, you can see that it's kind of diverged and reset its state to the new primary. So you'll lose that data, but you at least won't corrupt your database, which is a win, I think in my book. Any questions so far here? And lastly, what's that? Raphael has the question. Can you? That's a question. How big can the database be for this replication to work well? If you want to replicate your database to every edge nodes around the world? Sure. We typically would recommend, I mean, I would say we typically are targeting around one to 10 gigabyte databases. And beyond that, I find that, you know, especially with SQLite, you have a single writer. So you start limiting actual write throughput. So I see that it's, it usually makes more sense to start starting out your database. The nice thing about SQLite again is that you can have, you know, there's not really much overhead to have an additional database. So sometimes people build models where you actually have a database for customer, you know, as long as you don't have a million customers, if it's like a thousand customers, you can really get some great tenant isolation around having separate databases. And those tend to work better. So yeah, I wouldn't, I wouldn't recommend this for like a terabyte database by any means. That's going to hurt. Good question, though. Any other questions? So yeah, the point in time restores, you know, whereas before with Lightstream, we did this raw wall, raw wall file replication. We kind of introduced this new file format that's similar to the wall, but we can actually do a couple of interesting things. They support streaming of multiple of these files. So we can actually merge them together to get the latest version of each page. So you can actually roll up these kind of these wall files are called LTX files, these replication files into higher level files so that they're faster to restore later on. So you can get this kind of really instant restores for your database, which is it's nice for recovery, but it's also nice if you want to have if you're deploying out and you have like a staging area and you want to have a new staging area for each of your, you know, maybe your PRs, you could spin that up, you know, do a point in time restore from your database and then just start testing things out on that staging area. So there's a lot of really interesting things you can kind of do from a development standpoint with that. Finally, we got some other interesting pieces like encryption. We're going to be adding here soon. You know, if you push and stuff up to S3, you don't necessarily want S3 to have your data. So if you can encrypt it and do it transparently, you know, that's great. We're currently looking at ASGCM-SIV, which is kind of a new hip kid on the block for encryption, I guess, but it has some benefits as far as the nonce that make it pretty attractive for us. And then another issue, another thing too is with LTX is we have some associated event data. A lot of times we're replicating out to the edge nodes, especially with real time applications that need to get notifications of data changes. It's not just enough that the data changes underneath the application, but yeah, so we're going to be adding some events associated with event data associated with the transactions. So yeah, some interesting underlying file format changes. I think I'm kind of getting up on time here. I'll try to go through the rest of this kind of quickly here. There are alternatives. I don't want people to think that Lightstream is the only way and that it's the best thing ever. You know, everything has this trade-off. So just want to go through some of these here. Regular backups, honestly, is probably what most people need, like an hourly backup of your database, especially if it's a small database. You can really do wonders and it's really easy to do and it's really hard to mess up. And additionally, you can do this in addition to Lightstream so you can run the two together at the same time just for some additional safety. And yeah, we have even instructions on Lightstream's website for doing this. There's graph-based replication. There's tools like Arculite, which I know you guys have had Phillipo tool on I think last year to talk about this. He's great. And this gives you that really strong consistency, but there's trade-offs with that. You know, this is a bit more complicated to run and set up. You can't do that loose membership that, you know, with like ephemeral nodes, you really need to kind of have, every time you add or remove a node, it's, you know, some commands you need to actually make it, it's a little more complicated. That's all. There's some VFS-based tools out there. There's one from Backtrace Labs, which I butcher the name every single time. I don't know if it's Verneul. I don't know how you say it, but I think it's French. There's another one called Light replica, which is like an AGPL and a commercial license. I haven't used it before, but it's out there. And this uses SQLite's virtual file system. So they actually have a layer where they can intercept writes, but again, you have to load it up and actually compile it into your application. So it doesn't work for legacy applications. And then finally, Postgres and MySQL. Like I've done a bunch of apps with these, and they're great. It's kind of that traditional NTR architecture, and there's really nothing wrong with them. I mean, I think that SQLite has its place, and it's great, but I really don't want to knock these other databases. So, you know, a little bit more complicated, but you're going to get a lot of other interesting features, different currency models that you can use, not everything serializable. So you can get some performance. And lastly, I'll just say, like, don't rewrite an existing app and SQLite just for fun, because you saw this presentation. You know, there's nothing wrong with legacy applications and legacy tools. So keep on trucking if you're doing that. Okay. So the last point's here, just to cover everything real quick. Lightstream provides really like a disaster recovery for kind of these mid to small size apps. That's really its focus, and it's kind of going to stay its focus. You know, we're building light FS to kind of make this distributed SQLite ecosystem. And there's a lot of interesting stuff you can do around real-time replication and high availability.  So yeah, that's my talk on Lightstream today. And any questions? Thanks for the talk, virtual upload button. But yes, I have a question, but if anyone else has questions, they can go first. Can you hear me? Yep. So Lightstream is kind of blindly copying pages, like the redhead long pages. You're not actually parsing them and interpreting them on the way out. Have you thought about what optimizations you could potentially do if you actually parsed the wall? Sure. Yeah. So it does actually do some parsing of the wall. I should have covered that. So there's a wall header. So there's a wall header on the whole file and there's a wall frame header on each frame. So, you know, we can delineate transactions by that wall frame header. We can see which page is the last one in the transaction. That's how we can get transaction boundaries. And as far as actually parsing out the wall, the pages in the wall, there are some really interesting stuff you can do around that. It just gets really complicated. So there's something called CDC, which is change data capture, where you can actually, if you wanted to have a stream of all the data that changes in your database and you do an insert, or maybe like an update, you can actually parse out from the page and you could technically detect, you know, if a row is updated and what the previous version of the update is and the version after the update. So almost like a trigger, but external to the database. And there are tools like this for like, you know, PostgreSQL and MySQL out there. I can't remember the name of them now, but sometimes people use that. It's Debezium. He gave us last semester. Yeah, that's the guy. So you can use that. A lot of times people will pipe out those change data law or change data capture streams out to something like Kafka. So then you can keep a record of a lot of this stuff or like, it becomes kind of this enterprise service bus. I don't know if that term is still used anymore. I feel like that's like a Java term. But yeah, so that's, there's some really interesting stuff if you do actually parse the data. But the problem you get into that is you really need to like reconstruct the row and reconstruct, you know, that row, if it had moved pages and you'd actually find where it was in the previous page. And so there's some complexity there, but it could be definitely a fun little exercise. Awesome. Thanks. I don't know how many people are actually streaming out to change data from their SQL database to Kafka. That seems a bit heavy-handed, but it could, yeah. I guess speaking on heavy-handed, do you have some sense of like, let's say I was running SQLite before and now I run Lightstream. How much load does that introduce onto the machine? I imagine it's not actually that much, but maybe you have some estimates. Yeah. No, yeah. No, it's pretty minimal, I would say. I'd never seen it take out more than maybe 5% of the actual processing time. You know, the vast majority is going to be IO and a lot of the pages are all recently used, so they're all going to be in the page cache anyway. So that's super fast. And then beyond that, you know, it does have the LG4 compression. I would say that's the biggest CPU component.  So it's typically pretty minimal.