Welcome to Hard Questions where we gather pastors together to take on your tough questions and answer them right from the Bible. I'm Tom Hollis, the moderator and today our panelists include Dr. Ian Walglaze, Bethany Baptist Church in Pittsburgh. Pastor Buck Schaefer, Grace Life Church Monroeville in the North Hills. Pete Giacalone, South Hills Assembly Guide Church, Bethel Park, P.A. Janthony Gilbert, pastor of another level ministries in Mount Washington. Pastor, thank you for being with us today's topics, sex, marriage, and Solomon's many wives. We're going to get right after it here. Sometimes the biggest decision we have before we go on the air here is who's going to take the question. Who's going to take this question? Who's going to take that question? Bill, it's fallen to you to take this first question. What does it mean to be one flesh in marriage? Well, you know, it's interesting that three times in the word, Genesis and in Matthew and in Ephesians, it says, therefore shall a man leave his father and mother and cleave unto his wife, and the two shall be one flesh. Now, somebody said, as Jesus said, verily, you need to listen to that. If he said verily, verily, you need to listen. But he said this three times in the Bible. So, you know, we need to heed. So, what does that mean? You know, it says, therefore shall a man leave his father and mother. So, first of all, there's the turning away from relationships that were the most significant and most important to you. And then it says to cleave. And that word cleave actually means like superglue together. And then it says the weave that you become intertwined. And I believe that in the intertwining, the one flesh, it means several things. You know, become one sexually, become one emotionally. You know, become one financially, socially. So, I think that, you know, the one flesh means that your lives are intertwined with each other. And that's why divorce is so devastating.  And I think that that's what's happening, that one flesh, the weaving of the sexually, the emotionally, the socially.  So, I believe that's what that one flesh means. Yeah, you know, that's a great explanation. And I just want to ask, maybe Pastor Jay, that whole thing of when people are knit together like that sexually, that also argues for why premarital sex is a problem, because there's those same kind of ragged edges when people pull apart, when they're not together. Oh, without a doubt. I mean, there's a connection here.  You know, Marvin Gaye sang a song called Sexual Healing. There's something that happens when you come together, even in a marital union. When you come together, matter of fact, I don't know if you guys have ever done this, but I would talk and I do premarital counseling. I've married people that were sleeping together but chose to stop and then waited. And then I asked them, how was it? Now, I don't want no details, but like, what did they feel inside when they came together? Every single time they said there was a purity about it that I had never experienced before. So there's something supernatural that happens. Even if you're not saved, there's still something that's supernatural that happens when you come together with that person. Soul ties. All those things happen because God said, it's one flesh. And that's why I think he mentioned it so many times saying, hey, y'all might think it's just fun and games to get together and do your thing and get your swerve on as they say back in the day, but that's not what it's about. It's about coming together and becoming that one flesh. And that happens every time you're physically intimate.  No, I agree with Bill and everything that's said. You know, Malachi says in the Message Bible, I love it.  That's like it tears your part. It tears your kids apart. And that's where a lot of fences open up and a lot of hurts and pains where the devil destroys families, comes to kill, still in destroy. But I love it that when you're one flesh, God sees you as one. Your husband and wife, you are one. And really the goal is to be baptized and be one with him, one with Christ. So one flesh, very important. And that union is really a symbol of our relationship with Christ. And Peter, I want to get into the second question because this just flows right into this. It does. And the second question is this. What does the Bible say about sex before marriage? And specific scriptures as well. They did ask Ephesians 5, 3-5. It says, But fornication in all cleanliness and covetousness, let it not even be named among you as fitting for the saints. Now again, we have to distinguish between the nonbeliever and the believer. But for those who are born again and as a pastor, I'm seeing a lot of this these last two decades that we have allowed the lifestyle of the nonbeliever to enter into the believer's life where believers before marriage are living together. It says, Nor foolish talking nor chorus jesting, which are not fitting, but rather giving of thanks for this you know that here we go. No fornicator, unclean person nor covetous man who is an idolater has any inheritance in the kingdom of Christ and God. So in other words, are we really giving full credence to the Word of God? Also in 1 Corinthians 6, 18 through 19, flee sexual immorality. Every sin that a man does is outside the body, but he who commits sexual immorality sins against his own body.  So we're told from the very gikko, yes, the passion, we're going to get later on to talk about Solomon, the passion is there. You know, when I fell in love with Elaine, we made a vow together that our honeymoon night would be our honeymoon night and our honeymoon night was our honeymoon night. I've had people say, oh, come on, do you expect us to believe that? I said, I don't care if you believe it or not, I can stand before God and know that nothing happened and I'm grateful. Now, does that mean that nothing happened in my life before Elaine? No, unfortunately. And I regret that. I know that's under the blood. And I did not want that sin to be part of our relationship. And I know Doc has some quotes about those who do live in sin. No, to me the most, and I appreciate those verses that Pete brought out, but to me the most powerful verse, and when we look at fornication, that sex outside of marriage, and they're asking specifically, is Hebrews 13 for marriage is honorable and all, and the bed is undefiled, but fornicators and adulterers, God will judge. So, you know, God is saying that marriage is the way to go. Marriage is my plan.  You can knock yourself out sexually within the marriage boundaries, right?  That's the glazes telling us to knock ourselves out sexually here. In marriage, of course. In marriage. Well, let me say real quickly. You guys have done well covering that. The reason why too, I believe fornication is bad, especially if you're dating, it robs you of developing the foundation in the relationship. If the sex is good, you don't need to get to know the person. And you can be so caught up in the intimacy of the physical part of it that you forget to talk about what's really important. That soul connect that needs to happen. Are we compatible? Do we see things eye to eye? So if the sex is good and let's just be real, then what happens in a relationship when you're dating somebody, you're like, oh, I don't really matter. Oh, I'm just so in love. They're just so beautiful. They're just so wonderful.   You are not even compatible. And then when that sex gets old, you don't have the ability to have a sustaining, lasting relationship. But listen to the way it says it in the Amplified. It says, That's pretty clear.  No, it's not the idea of being a Bible thumper. It's the idea of being a watchguard saying, be careful. This will take you down the wrong road. This will take you down to a path of destruction, even to those who are believers. That's right. You mentioned counseling. I know someone who counsel couples and he said, you know, people, what do they say in the world? Well, you have to experience this. Try it out. Try it out. Or work. He said, every time it caused hardness. Yes. Whereas those that did not, there was an openness. There was a softness. Whereas the experience, so to speak, caused a hardness and difficulty in that relationship. And real quick, if the car's really, really valuable, like a Bentley, you don't test drive Bentley's. No. You buy them. You can't even touch them when you go on the lot. Don't touch the Bentley, everybody. Come on, somebody. If you're a used car, another story. All right, well. Take it out on the bed. Coming up in 60 seconds, we ask, why was Solomon considered a golly man when he had so many wives? We'll be right back. Well, thank you for coming back to hard questions. We've been talking about sex, marriage, all those things. Pastor Jay, I want to get your thoughts and your biblical perspective on what does the Bible say about divorce and remarriage? And how is that different from what the world says? There's a lot to that. So I'm going to be kind of quick and share the wealth here. Well, you know, the Bible, I believe there's three major reasons why you can get a divorce biblically. Adultery, abuse, and abandonment. And the scriptures that I say about, there's one with Jesus talked about how except for adultery or fornication, that person is not allowed to get a divorce. And then also in 1 Corinthians 7, it talks about if the unbeliever chooses to depart and they choose to abandon you, then that's another reason. Then I think the abuse part for me is this. It's not the fact that if anybody ever made a mistake, can they be? But if someone's constantly abusing somebody, then I believe this is my personal take on it. You apply the Matthew 18 principle to it. Because then you got to wonder if they're even a believer. And then if you choose to back away, most of the time they're going to be choosing to leave and go their way as well. And most of the time, that's when you're going to see a divorce happen with that as well. Now it's not any one of those three though. Let me say this. You don't have to leave. You never have to leave. But I believe the Bible gives us an opportunity to leave. And if we leave under those bad grounds, the Bible forbids that we're not allowed to remarry. Okay. I think if you're a believer and you walk in 1 Corinthians, Agape, 13, God kind of love, the love of God never went to a divorce court. But on the other side of that, you have people that want to get a divorce for whatever reason. I agree with Jay. Abandonment, abuse. We've dealt with tons of couples.  And it's just like, they keep doing it, keep doing it. Well, the Bible sets this up because Moses said, someone's heart's getting hard. And the law of Moses made a way for this.  If you get a divorce, it's like the unpardonable sin. Yeah, right, right.  And I disagree with that. I feel like you put that which is behind you. You get up on your feet. God loves you. Do the will of God. But it's not something we do every other year. Yeah, yeah, absolutely. And actually that was a problem in the Roman world. I don't like getting divorced and remarried like every year. But let me ask this side of the table. What about the remarriage side of that? What are the pastor glades? You first. What do you think on that side of it, biblically? Yeah, well, I think that if you don't have a biblical reason for divorce, then you don't have a biblical reason for remarriage. And you know, that's hard. You know, and I get a lot of people, and I'm sure these other pastors do too, that call me. And they're just struggling in their marriage. And the first thing that they ask me, you know, if I get a divorce, can I get remarried? And I'm like, well, you need to deal with your marriage right now. And if you walk away from it and you don't have a biblical reason, then you don't have a biblical reason for remarriage. But I'm seeing that people are getting remarried anyway. So, you know, and then some people will tell them, okay, well, you're in adultery. And now that you've got remarried without a biblical reason.  So, you know, I would struggle to say that you're in a continual state of adultery, which is what some people will tell them. Wow. And remember, what they're leaning on here was, it was because of the hardness of your heart. You know, if dinner wasn't made properly, they handed the wife a bill of divorce.  And you know what? It's coming to that this way today. And I really believe if couples, in their premarital, I'm speaking to the believer now, the nonbeliever, I can't help them with what I'm going to say. But for the believer, if their marriage is built on the word, before marriage, if their life is built on the word before marriage, you've got two solid believers coming together, then you've got a solid foundation. And it comes back to this word. It comes back to the word of God once again. You're talking that Bible?  I know we need a lead right into the next question. A lot of people glance over this in Matthew chapter 1 and verse 6. I think it's very profound. It says, in Jesse Begat, David the King, and David the King Begat Solomon, the wife of Uriah. I think it's outstanding that Uriah was dead. He married Bathsheba, but the Bible, the heaven did not recognize that marriage. So even though earth might have the papers, heaven may say null and void. So I think that's something else that's important as well. Wow, that's powerful. I think it's one of the things, this is what we face, right? You alluded to it, is that we know people that are divorced and remarried, godly people, loving the Lord, and ministry, many of them. And it almost seems like, wait a minute, is that the biblical model for that? It's hard for us to figure out how to go forward with that. But we do because they're good people and they're the servant of the Lord. There's powerful pastors that in the beginning, I don't want to name their names, but powerful, powerful men that are literally reaching the entire world that their first marriage didn't win a ride. And they were the innocent party. I think that God always wants to err if there's such thing as air, but God wants to move on the side of compassion and forgiveness. Yeah, and grace. So the next one, we're going to talk about somebody who started off good and did not end so well. Why was Solomon considered the wisest man, godly man, when he had so many wives, 700 wives, 300 concubines? Tell me about this festival. Solomon started off good. He started off doing what his father wanted, but something that hit me is what Jay said. Maybe there's a little bit of that soul tie coming from a generational curse from before. But this dude was wise. This dude was smart. This dude had wisdom. He asked God for wisdom. So I'm having a struggle with one wife. So seven wives would be an issue. But 700 and then 300 concubines, I don't know what a man would do. But that's not wisdom at all, is it? So if you realize this guy writes proverbs, this guy writes song of Solomon about love and marriage and a spouse and how to treat her. But I think what happened if you study the scriptures, in the end, I think those wives turned his heart away from God. And if you notice, he was starting to do sacrifices for other gods. And the gods of his wives that had idols, I think he was probably just like, okay, go sacrifice the molek. And the Bible says he opened himself up to all this stuff. And I really believe that if you get the end of Ecclesiastes, he starts to try to correct that. And he says, it's all vanity and chasing of the wind. A man can have this and I did. A man can have that and I did. But this is all vanity, vanity. So I think he went out of the confines of what God wanted and God will let you do that. If you choose that, God will let you do that. But it did cost him and there was great consequences. And he walked away, Pastor Bill. Well, I heard somebody saying, I don't know how true it is, but I think there's some validity to it, is that Solomon wrote the song of Solomon as a youth. He wrote proverbs in his middle age and then he wrote Ecclesiastes in his elderly years. It seems like even though he let these women turn his heart away from God, that somewhere along the line, again, as Buck said, he realized what he did was wrong. So we can't use Solomon as a justification. As I heard somebody say, he has 700 wives and 300 porcupines. I was just thinking, if he had dinner with the one, it'd take him three and a half years to get back around. Get that out of your mind. But you know, when you think about it, you know, he learned. I believe he learned his lesson as to where he went wrong. And I think if you read Ecclesiastes, if he had the chance to do it again, maybe he might make a different decision.  He who commits that sin, sins against his body, you become whatever you yield yourself serving to, that's whom you will be slaved to. And that's the reason why if there's ever a day that we as born-again believers need to safeguard our mind, of course, through the eye-gate, hearing-gate, we need to, there's an onslaught war like never before. And if a wise man like Solomon could be, yes, we need to realize that. Real quick, I think it also shows us too that wisdom was given to him as a gift. And even though he was gifted with wisdom, his character still could have been faulty. So you can operate in a strong gifting and have a charisma, but still have lacking character. And I think God wants us to realize, no matter how gifted you are, there's still the part of you that's the humanity that we have to deal with as well. That's right, that's right. So those 700 wives, everybody, that's not gonna work out well. No porcupines either. All right, let's get to the task of how we answered a previous question. Stay tuned. Well, while we enjoy receiving your questions on Hard Questions Hotline, we also appreciate comments and we received this comment. Let's take a listen. Hello. This evening, while listening to Hard Questions, I don't think I heard wrong, but one of the questions was where they could find that it's wrong to have sex before marriage in the Bible. And the answer that was given was really surprising that there really wasn't such a place. Now, I use the NIV version, have been using it for decades, and I know it's very clear, Lee answered in Ephesians 5, chapter 5 verse 3, or 1st Thessalonians, chapter 4 verses 3 to 5. It couldn't be clearer there. Galatians 5, 19 and 24 are the verses there. And 1st Corinthians 5, the first verse. I hope I did misunderstand the question because it really did seem like that was the answer that they were looking for. Thank you. Bye.  I really wrestled with this because I don't think we would ever say that there's nowhere in the Bible that says sex outside of marriage is wrong. There certainly is. Pastor Jay, you had a comment about this. Yeah, I was thinking that there are times we've discussed that and we've mentioned about how the battle between, like someone says, it's living together wrong. They call it shacking up, living together a man and a woman. Well, that in itself is not necessarily a sin. Now, you could go to the case of avoid the appearance of evil. We get that, but it's not sexual sin to live in the same house. Sexual sin. And I'm wondering if that's where this individual took that stance. I'm with you. I don't think anyone's ever... Although almost 100% of people that are living together are sexual activity going on as well. And I totally... And I wouldn't condone that. I just wonder when we are addressing that if that's where they took that and went with it. But I know even with what you said, I don't think anybody's ever taken a stance that that's not biblical. And I know that as pastors all of us have run into it because I remember a specific situation where I was counseling a couple in premarital counseling and they were living together and they actually had, like, three kids and they were like 12 years old. But I did recommend to them that they separate and sleep in separate bedrooms. And they came back to the next session and the lady was upset with me. She said I can't believe he moved out of the bedroom. And he moved out of the bedroom and so they were in the same house but they had so much connected and so many things hide together that they couldn't actually move out so they moved in separate bedrooms. And I remember the lady told me she said, Pastor Blaze, she said when you marry us, she said we're not coming straight to the reception. That we're going to stop, stop at the house. Oh my goodness. And so I married them and the day came and everybody was at the reception and they didn't show up, man. And I said to myself, I know I wouldn't stop them. Well, Pete, how about some verses of the news show? Now you want to drag me in. Now you want to drag me in. Get us out of here, Pete. How about some verses? Well, again, I think the scriptures she quoted and the scriptures we quoted just a little while ago it comes back to are we honoring God? Amen. Am I honoring God with my body?  Because, again, if we I'm going to thump it again back to the principles of the Word of God and she read all the correct scriptures and I know where Jay is coming as a pastor that there may be some situations where a couple should live together. Again, all of us would say don't even go there. And there's no way I know you Jay well enough. There's no way that you're promoting intimacy. There's a lot of variables involved. Well, that's where our society is. I mean, I had someone call on the prayer line when I was answering here one time and said they wanted all this healing in their relationship and they were living together and I said, well, you know, God's not going to bless that. And she said, wait, you mean I have to move? I can't live together. And I said, well, yeah. And she hung up on me. In this world we live in in this age, the spirit of this age in our culture, they're all living together. So I think it's very careful that we come about this because once I bring the truth to people that want to get married and they address that truth in their hearts, they do, they separate, they move out. I make the recommendation. But that's where faith begins where the will of God is known. Once you know the will of God and obey that, things turn for the good. Absolutely. That's a good commentary and sorry for the confusion. If there was any, we certainly believe God wants that protected environment for that closeness, for that relationship, for that to be protected. Well, we'd like to end the program with the Scripture and today we go to Romans where it says be devoted to one another and love, honor one another above yourselves. We hope that you've enjoyed today's program and we want to hear from you. Email us your questions at hardquestionsatctvn.org or call into our hotline at<PHONE_NUMBER>.


Hi and welcome back. So far we have created a teacher account and we have explored the teacher dashboard and teacher tools available on the Khan Academy platform. Now let us think about it. When a teacher joins a school, what are the fundamental tasks that they need to do? Teachers get a class to teach and in that class, students are added. Similarly, now that you have joined the platform, we will learn how you can create a class and add students to your class on Khan Academy. When you log into your teacher's account, you will land up at the home page, which is called the teacher dashboard. We begin by clicking on add new class here on the top right. You will see two options on your screen. You can enter a class name of your choice and make sure it helps you easily identify the class. We would recommend you name your class with the grade you are teaching and the academic year example 6th A 2023-24. Click on next and then select the subjects you feel are most relevant to this class. It's advisable to select content from a few standards below and higher to allow for remediation and engage advanced learners. Let's say you are a teacher for 6th grade math, then in addition to 6th grade math, you may want to choose 5th grade content to help you with remediation and 7th grade content to engage the advanced learners. You can add or remove the subjects at any point. There are a lot of subjects, so remember to scroll down. Smartphone users can follow the same process to add a new class. In case you are already using Google Classroom, you can click on import your class from Google Classroom. Please select your class from the drop down list and click next. You will see a message on your screen that an email invite has been shared with your students. You can ask your students to check their email and join your class by clicking on the invite link. In case you are not using Google Classroom, please don't worry. In the next section, we will show you other ways to add your students. Now that your class is ready, it's time to add your students to this class. Students can be added in 3 ways. If your school is using Google Classroom, it is better to import the entire class while creating your class as explained earlier. If your students are already using Khan Academy, you can share your class code link with them and they can add you as a teacher. You can also get the class code by clicking on students. You will be able to see the class code at the top right of the student list. Smartphone users can find the class code at the bottom after clicking on students. If your students are logging into Khan Academy for the first time, we recommend that you create their accounts. All we need to do is enter the student's name from your class and the platform will auto generate a username for them. The username will be unique for each student. This username cannot be edited. In case you have multiple students with the same first name, then we recommend that you update the full name of the students. Once we have created the user names, you need to click on the next button. After clicking the next button, the platform will auto generate the passwords for each student. However, the auto generated passwords are not easy to remember. So we recommend that you change the password to something that can be easily remembered by the students. You can use the same password for all the students as well. Smartphone users can follow the same steps. However, you may have to scroll to the right to see and change the passwords. Remember, it is important that you change the password for students at the time of creating their account. Once the accounts are created, there is currently no option available to change the password of all students at once. However, you can always change the password individually for each of your students. While creating the accounts, please remember that this content does not get saved till you click on create accounts. So if you were to exit it midway, you would lose all your data and would have to enter it all over again. Also, please do not click anywhere outside the account creation window as doing this closes it and you would lose your data. We recommend you create this in batches of about 20 students each to prevent loss of data. Once you have edited the passwords, you need to click on create accounts. You will get an option to download the student user names and passwords as a CSV file. This is a very important step as you can download this list only once. Make sure you click on download CSV and make multiple copies of this list. Have it on your email, your mobile and a hard copy of this in your class. Students often forget their user names and passwords and having this list handy is helpful. You can add students to this class as many times as you wish and as many students to the class. There is no limit. Very often, students may join the class in the middle of the year as well. To add these students, select the class to which you want to add them. Click on students and click on add new students. You can create a new username and a password for them or if they were previously using Khan Academy share the class code with them and they can add you as a teacher. Please note that creating your class on Khan Academy platform and adding students is an important task as this will link the students accounts to your teacher account and therefore allow you to track their progress and scores. We would recommend that you create your class and add students to it before you move ahead with the next lessons in this course. Once you have added all the students to your class by creating their accounts, please ensure that you share the user names and passwords with the students. You can share the entire list on WhatsApp or you can print and paste it to your class so that your students can refer to them anytime.


Switch it off, no, that's okay. Thank you. So, good morning everyone, welcome. So first of all, I would like to thank the organizers for inviting me. I would like to talk to you about joint work or actually two papers with different subsets of co-authors, Dennis Elbrechter, who is a PhD student of Philip Grossis who is a professor in Vienna, was formerly at ETH. Gitta Kutinyuk from TU Berlin, Philip Pedersen who was Gitta's student, and Mitro Perechristenko who is my PhD student, and who also generously helped me with the slides. So, you didn't think you could just sit here and listen, right? So I'm going to make you work a little bit. We just heard Jared talk about classification. So here is a classification problem for you to solve. Can you help me annotate the images? So there's four groups, and each group contains three images. So any suggestions for any people you recognize? Don't think science only. Dark chocolate for those that make correct guesses after the talk. Joachim. Gödel and Karajan, yes? Then there are two more. von Neumann, yes? So the lady as the color suggests is more recent. Not science. Any opera music lovers here? OK, I'll show you. This is Elina Garancia with the famous Mezzo Soprano. Another problem, image annotation. So this is what deep neural networks excel in, classification, image annotation. Can you help me annotate this image? This is to help you a little bit. He's considered one of the greatest conductors of the 20th century and at the same time was the least prolific. So von Karajan said that he conducts only when his fridge is empty. Any ideas? You probably recognize the background, no? Or the concert. It's a famous concert that you can listen to every year. The New Year's concert, yes? Sorry, in Vienna, yes, right? But we have to start with the name of the conductor. You see, I think this just proves. You can be a genius without being widely known. So if you haven't heard about Carlos Kleiber, I recommend looking him up. Everything he ever conducted is on 11 CDs. And he was known to be somewhat special. So he would, for example, agree to conduct the Vienna Philharmonics, not for the New Year's concert, but for a regular concert. And then he would come and do rehearsals. And then he would decide the day of the concert that they were not in good enough shape to be conducted by him. He would just leave a note saying, drove into the blue Carlos on the day of the performance. And they would then struggle to find a replacement for him. Good. Conducting the Vienna Philharmonics New Year's concert. And this was the experts. He conducted it twice. Conducting the New Year's concert is like winning a Nobel Prize. And so there's two years. You can actually, there's enough information in the image to tell a part. If you're interested, go to YouTube. He has two concerts. And one is in 89. And the other one is in 92. And the way you can tell is by the flower bouquets in the background and the colors. So this is 89. And 92 was yellow and green. OK, good. But let's get serious now. I also do have technical stuff to talk about. So what I would like to talk about is neural networks, mappings, concatenations of affine transformations, nonlinearities, affine transformations, nonlinearities, and so on. So WL is an affine map. Then we have what is called an activation function, rho acts component-wise. The quantities to keep in mind in this talk are the network connectivity, which is the total number of non-zero parameters in those affine transformations. And defines the topology of the network, the number of layers, or the depths of the network, L, and the width of the network, W. So the most relevant quantity for us will be M, the total number of non-zero parameters in the WLs. So what I would like to do here is I would like to isolate one question. And that's the question of what can we learn fundamentally with the neural network? And the way I'm going to do it is I'm going to assume that we have, I call this, access to infinite amounts of data. But what it really means is we know the entire function that we want to learn. So you will say immediately, well, of course, this never happens in practice, yes. But this talk is somewhat information serrating in the sense that we isolate certain aspects and we ask what is possible at all. So we know the function. We have the optimal learning algorithm available. We don't care about the learning algorithm. So this talk has nothing about learning from finite amounts of data. It has nothing about learning algorithms. All it has is it asks, which classes of functions can be represented and represented efficiently as a concatenation of linear transformations and nonlinearities. And what I mean by efficiently, I will tell you in a second. So for those of you who are familiar with Sibenko and Hornig, you will immediately say, well, wait a moment. What's the point of this talk? We know that any continuous function can be represented through a single hidden layer neural network. And that's a result that dates back to 1989 for sigmoidal activation functions. So that's correct. What it does not tell you is how the width of the network scales as a function of epsilon. So if you want to increase the precision, how does the size of the network grow? There's no result there. There is a more recent result by Louis et al that tells you that an integrable function mapping rn2r can be represented through a real network of width n plus 4 with an error of no more than epsilon. And here there's no bound on that. So there's these two approximation results. And then there is a whole universe of follow up results starting with the papers by Andrew Berron in 93 and 94 that gave approximation error bounds for smooth functions. And I'm not going to read out all these references to you. And it's highly incomplete, this list. There's literally hundreds of papers that were trying to answer the question of the complexity of these networks for specific function classes. What I would like to point out is results from the 1990s by Charles Chewy and also Marshcar that observed that in certain cases deep networks can perform better than single hidden layer neural networks. And there's going to be something in this talk about what deep networks can do that shallow networks cannot do. So there's going to be some precise statements in that. And more recently, Eldan and Shamir, there exist functions that are expressible through small three-layer networks but only through very large two-layer networks. So these are all observations that speak in favor of depths. Some more recent work I would like to mention by colleagues from Zurich, Arnulf Jensen, Christoph Schwab, and then Philip Gross that showed that deep neural networks can break the curse of the nationality in approximating or in solving certain PDEs. The paper that inspired us for this talk is a paper by Shaham Cloninger and Coifman that talks about approximations with wavelet basis and connects this to neural networks. So the paper, it's not very explicit, but the main philosophy of our talk is somehow quite hidden in that paper. But anyway, so I saw that paper in archive and this is what God has started. So I would like to encourage you to have a look at that beautiful paper. Good. So obviously, all these references, they tell you that there should be some kind of systematic framework that asks for the complexity of the approximating network. In particular, we will be looking at two different cases. One is the approximation of individual functions. For example, the first one that we'll look at is the squaring function, then the multiplication function, polynomial, smooth functions, sinusoidal functions, and so on. And in that context, approximation of individual functions, we are going to be interested in the number of nodes in the network needed to achieve a certain approximation error. In particular, we are going to be interested in what is called exponential approximation accuracy, the approximation error, the case exponentially error in the number of nodes in the network. And the main statement or the main result will be that deep networks provide exponential approximation accuracy for a wide range of functions. In fact, I was tempted to put here for anything you can think of, but then I wanted to be a bit more conservative. OK, and the second part of the talk is going to look at the approximation of entire function classes. So for those of you who were in the 1990s and early 2000s in a whole wave of approximation theoretic results, wave of space, and so on, they know about those results that wavelengths optimally approximate functions from base of spaces. And then there is many other results. And I'm going to talk about that framework. And we are going to be interested in neural networks that approximate functions from a given function class. It's going to be a worst case theory. And we're going to ask for the complexity of the network. And here we're going to be much closer to reality than in the first part. We're going to quantize the network in the sense of we're going to ask, what is the number of bits needed to encode the topology of the network? And you're going to quantize the weights and the biases in the network. And you want to store the entire network in a bit string. So I give you a bit string from that bit string so you encode the network into that bit string. And then from that bit string you should be able to uniquely read out the network, reconstruct it. So the network is going to act as an encoder for an element of that function class. And then immediately you will start thinking about Komogorov, epsilon entropy, covering numbers, and so on. And indeed, there is a deep relationship here. Good. OK, so let's start with the approximation of the squaring function. So the result is as follows. You can approximate the squaring function with a deep neural network, with an error of at most epsilon, with a network with a width of four, and the depth scaling logarithmic in the inverse of the approximation error. Here this is a bound on the size of the weights. The size of the weights will play a very important role later on, because when we talk about Komogorov, actually it's going to be Komogorov-Donohoe optimality. The scaling of the weights in the domain over which you want to approximate the function is going to turn out to be crucial, but more on that later. Good. So the total number of parameters in this network is 4 times c times log 1 over epsilon. So what that means is that, as you will see in a second, that we are going to get exponential approximation accuracy. But before I show you this result, let me just tell you about Yarotsky's work, who had this result on the approximation of the squaring function. So we have some minor improvements or modifications of that result, but this is the paper you should be looking at, Yarotsky 2017. Maybe the two things I want to mention here is that relative to his result, the construction that we have is we get the weights of the network to scale no faster than polynomial in the size of the domain over which we want to approximate the function, which we need for optimality later on. And the second aspect we need is in approximating polynomials, in contrast to Yarotsky's work, the width of our network does not scale in the degree of the polynomial. We're going to use a Taylor series argument later in approximating sinusoidal functions. And if the width would scale in the degree, then this wouldn't give us optimality. So these are the two things that may be new here. All right, so finite widths for depth scaling like log 1 over epsilon. Now the total number of non-zero weights in the network is upper bounded by this quantity here. So what is this? This is the depth times w squared. Well, this is the size of the matrices in the affine transform, w squared, and then plus w. This is the dimension of the bias vector, so L times w squared plus w. And if you stick in 4 and log 1 over epsilon and you rewrite the equation, you get that the approximation error decays according to 2 to the minus, some constant connectivity. The p is the polynomial degree we'll later have instead of log 1 over epsilon, polar log 1 over epsilon. But here p is equal to 1 in the approximation of x squared. So the finite widths combined with polar log and 1 over epsilon depths yields exponential error decay in connectivity. That's going to be relevant late in the talk where I will show you two function classes that are known to be very hard to approximate in classical approximation theory, namely the Weierstrass function and oscillatory textures. And the best known approximation procedures, algorithms give you a polynomial error decay. And it turns out with a deep neural network you can very easily get exponential error decay. Good. So the slide is titled Optimal Neural Network Algebra. Why algebra? Well, once we know how to approximate optimally, optimally meaning this exponential error decay, the squaring function, we're going to take this to approximate the multiplication function. Why the multiplication function? Well, once you have squaring, once you have multiplication, you can build polynomials. Once you have polynomials, you can do pillar series approximations, and so on and so forth. All right, so multiplication. We write the product x times y in this form here, y. Well, we know how to square. We know how to form linear combinations. Well, we have affine transformations available. So this is actually just a composition of three neural networks. One computes x plus y squared. The other one x squared. And the third one y squared. So you can show that optimality is preserved. And that's how you build multiplication. And then you go on, and you realize x to the k through the combination of squaring and multiplication operations. And then you can build smooth functions through Chebyshev interpolation combined with polynomial approximation. And finally, you can build sinusoidal functions through a Taylor series approach. And there is actually a twist that is important. And this is actually, I held back and I didn't talk much about this, but this is where we fought the hardest with the problem. Because we needed, in order to get optimality for the strong optimality result that I'm going to show you later, we needed optimality for the approximation of sinusoidal functions. And so this is, I would say, 80% of the research time that went into the paper was spent on this problem. But never mind. You can read it up. It's done now. In the end, it's not so difficult. But what it uses is actually the symmetry of the head function that you can build from a reload, so a function like this, a linear combination of reloads, shifted reloads, and together with the symmetry of the sine and the cosine. All right. So deep neural network approximation of function classes, we want to establish a relationship between the complexity of the function class c, compact z, and the complexity of corresponding approximating networks. Well, first, we have to say what we mean, or we have to define complexity of a function class. Well, as I said, you will think of Kolmogorov covering numbers and what we mean by complexity of a network. Well, the network is going to be encoded by a bit string, as I told you. It's going to, this bit string is going to encode the topology of the network and the quantized weights. Why do we want to quantize the weights? Well, because we have to quantize in practice. I mean, you cannot use infinitely many bits to represent real numbers. Good. The framework we are going to use was introduced by David Donahoe, based on a beautiful work by Kolmogorov. The paper that I learned this from is this one here by Ko and Dama and Dupshi and Dvor in 2001, where this is looked at in the context of certain wavelet-based encoders. And the basic idea is this. You have a function class, which is a compact z in L2 of omega. You have a set of encoders, or you have valid encoders, and encoder would map an element of C onto a bit string of length L. So just a sequence of zeros and ones. And the decoder would take this bit string and demap it back into function L2 of omega. So you think of Kolmogorov covering numbers. You encode the ball centers. And then you know the radios. You know the maximum error. So this is kind of philosophy that is behind this. We are going to use the minimax code lengths, which is the minimum code length such that there exists an encoder-decoder pair so that the worst case error is no more than epsilon. And in particular, we're going to be interested in this minimax code lengths scaling as epsilon to the minus 1 over gamma. And we're going to look at the largest gamma. So larger gamma is better because it gives us a smaller growth rate. I mean, growth rate, I mean, when the error goes to zero. So if you want the more and more precise approximation, then you're going to ask yourself, well, how does the number of bits needed to encode those functions scale when epsilon goes to zero? So larger gamma is better because the requirements for storing signals in C are more lenient. Good. OK. So here is the relationship to Kolmogorov-Tikomir of epsilon entropy.  You take the log of that number. So that's your Kolmogorov-Tikomir of epsilon entropy. And this number depends, of course, on the radius epsilon.  And it turns out that when C is not a finite set, then your h epsilon of C is going to go to infinity. As epsilon goes to zero, the question is, how does it go to infinity? And in many interesting cases, it goes to infinity according to epsilon to the minus 1 over some alpha, potentially also with a log term here. So in this talk, I'm going to ignore a log term or whatever have it there. So I'm just going to look at this quantity here. I'm going to look at the optimal exponent. And so this is a crude measure of growth, if you like. So whenever we talk about optimality, we mean modulo whatever you have here that grows more slowly than epsilon to the 1 minus 1 over alpha. Good. All right. So now switching gears. So where was this framework used very successfully? It was used in dictionary approximation. What is dictionary approximation? If I give you a function class, for example, base of spaces or modulation spaces, so those are spaces that relate to the short and Fourier transform, and give you a dictionary. For example, wavelets or Gabor dictionaries in physics coherent states, then you try to find linear combinations in those dictionaries to approximate your functions. Well, so that's dictionary approximation. And the question that you can now ask is the following. Suppose you have a dictionary, a set of functions, and you want to represent any function in your compact C as a linear combination of elements in that dictionary. And you refer to those approximations as best m term approximation. Why?  And in particular, you're going to ask for the error to decay like m to the minus gamma and for the largest exponent so that you can get this error decay. And that exponent is going to be denoted by a gamma star of C comma d. So clearly, this exponent depends on the function class, but also on the dictionary. In particular, it depends on how well the dictionary is matched to that function class. And that matching problem has been investigated very intensely some 20 years ago. And I'm going to show you some of the main results later on. But before we do that, let's ask ourselves what is if any, there is a fundamental limit on gamma star of C comma d. So you fix your compact set. And you are allowed to vary over all possible dictionaries. So this is the problem of dictionary mapping. What is the, is there an upper bound on gamma star of C comma d? And the answer is no, really, because if you take a dense and actually countable d, you can get gamma star to be equal to infinity. So you can approximate arbitrarily accurately. However, this is not terribly practical. Why? Because if you have a dictionary of infinitely many elements, you have to search into the dictionary infinitely deep. And you need an infinite number of bits to store the indices of the participating elements. So that defies the whole purpose. So what, a way around, this is a concept that David O'Neill introduced in 93, who defined this idea or this concept of effective approximation. So what he said is that if you want an m-term approximation, you're not going to be allowed to search the entire dictionary. You're going to be allowed to search the first pi, where pi is a polynomial, pi of m elements. So only polynomially deep, you're not going to be only polynomially deep into the dictionary. And you can already guess what this is going to do to the number of, so first of all, searching is feasible. Second, the number of bits to encode participating indices. Well, if you have a polynomial in many elements, a polynomial in m, so then the log of that is going to be some constant times log of m. And so you can actually encode with log m bits.  So it's m log m. I'm going to show you later. The second thing is we're going to need a bound on the coefficients because we want to quantize them. And this will also show up later in your network approximation. What you need to do here is you need to control the error that you incurred by quantizing the coefficients in your representation. And there's literature on this, if you're interested. The way you do this is that you simply show that this is actually a continuous mapping here, the superposition. And then if you quantize accurately enough, then you can show you can orthogonalize the participating elements, sorry. And then the approximation error gets preserved because the condition number of the orthogonal matrix is equal to 1. These are fairly standard things. And I'm mentioning this because in neural networks this is going to be a bit harder because we have to deal with nonlinear mappings. All right. So here is a don't host concept of effective best m term approximation rate. You want to have a dictionary approximation m terms. You want to have a decay of m to the minus gamma. You want to have a bound on your coefficients. And you're asking for the largest exponent, which we call now gamma star effective of c, d. And you refer to this as the effective best m term approximation rate. So here's the question now. What, if any, is the upper bound on gamma star effective of c, d? And what gives this gamma star effective of c, d, operational significance is that indeed you can show it's upper bounded by the Kolmogorov exponent. And the question that we are going to ask ourselves now is what are dictionaries for a given function class that optimally represent effectively that function class in the sense that the corresponding dictionary for this given function class achieves the Kolmogorov exponent. And this is what I will refer to as Kolmogorov Donohoe optimality from now on. And you can already see where the talk is going to go. So I'm going to show you now what kinds of results are available for classical approximation of function classes. And then we're going to ask ourselves, what are the function classes that neural networks optimally approximate in the Kolmogorov Donohoe framework? Well, we cannot apply it quite because neural network approximation is something different. So we're going to define this concept or the theory of, say, extension of Kolmogorov Donohoe approximation for neural networks. And then we're going to ask ourselves, what are the function classes that neural networks approximate? The neural network, you can then say acts like a dictionary, and we're going to find the c's that neural networks approximate optimally. And the answer to this question will be essentially anything you can think of. So that's a very strong universality result. The neural networks are Kolmogorov Donohoe optimal for essentially any function class you can think of. Good, but I will need some more time to get there. All right, so a back of the envelope calculation. Why the choices that we made lead to optimality? So polynomial depth search, meaning log m bits, to represent the indices of the participating elements. You have m elements. So you have order m log m bits to represent the indices of the participating dictionary elements. You quantize by rounding to integer multiples of m to the minus gamma. Why? Well, if you have the entire range bounded, then you divide by m to the minus alpha, sorry. Then you have a constant times m to the alpha. You take the log thereof, and that's log m again. And then you have m different elements. You have m log m bits to represent the quantized coefficients. And then you can control the error that you incur through this quantization. All right, and then you can show that there is an encoder decoder pair reconstructing f from order m log m bits with epsilon scaling like this here. And if you have an optimal dictionary for a given compact c, then you're going to get m log m to behave like epsilon to the minus 1 over gamma star times this term here, which we write this as o tilde. So this actually ignores the log, and you get the optimal exponent. OK, so that's kind of the philosophy that we're going to follow. We are going to be interested in d being replaced by neural networks, and then we're going to ask what are c's that give you Kolmogorov optimal approximation for neural networks. OK, but before we get there, here is an incomplete summary of what is known in terms of optimality results for given dictionaries or pairings of dictionaries and function classes. So this is LP. So we'll have Halder, Bump, Algebra, Bounded Variation, Base of Spaces, Modulation Spaces, as defined by Feichtinger and Grochanik. These are the optimal dictionaries. Here in square brackets, you have suboptimal dictionaries. So for example, here, let's say base of spaces, wavelets, Fourier is suboptimal.  So it's really important, of course, to choose a matching dictionary. So as already indicated, at the end of the talk, you will see that so here is, if you give me a function class, let's say any of those, then, or if you give me a data set, and so I would view this as you giving me a function class, then I would need to know something about the structure of the data. Is it in a base of space, in a relation space? I mean, what kind of smoothness do I have and so on? Then I go, I pick the optimal dictionary, and then I deploy my approximation algorithm. However, given what I said before, if you have a neural network, I don't have to know anything about the structure of the data. I just deploy it with the optimal learning algorithm, which I cannot specify, and I do not specify. But the neural network is going to find it, and in some cases, actually, we can prove that it finds the optimal dictionary even. Those are stylized cases. I didn't want to put them up, because I don't think these results are strong enough to be shown here. But they universally approximate all those function classes, optimally, in the sense of Kolmogorov-Donohoe. So what is it that we need to do in terms of modifications if we want to take this idea or this concept to neural networks? So we are going to replace m term approximation by m weight approximation. So the number of participating dictionary elements is now replaced by the number of non-zero entries in your weight matrices. So the connectivity of the network plays the role of the number of participating terms. We are going to have to encode the network topology and the quantized weights, and we will need to control the quantization-induced error. So that leads us to this concept of best m weight approximation rate. And again, we're going to let the error decay like m to the minus gamma, and we're going to look for the largest gamma and refer to it as the best m weight approximation rate. So there are two things that I talked about in dictionary approximation. One is polynomial depth search. Do we need something equivalent here? No, we don't. Why? Because the tree-like structure of the network already makes sure that there is no problem there. How do you see this? Well, what is the total number of non-zero weights in the network? You cannot exceed the depth times w squared plus w, obviously. So if the connectivity is m, neither the depths nor the weights can scale faster than m. So therefore, what we get here is a scaling of at most order of m cubed. Now, how many bits do you need to encode m locations of non-zero weights in a set of m cubed locations? You need log m cubed choose m, which is order m log m bits. So the tree-like structure naturally gives you already what you enforce in dictionary approximation through polynomial depth search. So that's one. So we don't have to care about that. The second one is the conversation of the weights. And that indeed I'm not going to show you the results. You have to work for this. The way you do it is you exploit Lipschitz's continuity of your non-linearities. And then you have to come up with error estimates. It's messy, but it's not very conceptual and not very difficult, but it's just a lot of work. So I don't think it's interesting enough to be shown here. So how do we let the depth scale of the network? Well, inspired by the approximation of individual functions, we let it scale like log 1 over epsilon. I mean, we have different choices here. It cannot scale faster than m, m is the connectivity. Since our error is always proportional to m to the minus gamma, we're going to have the depths grow polylogarithmically in m. OK, good. So that, in summary, gives us best embed approximations subject to polylogarithmic depths and polynomial weight growth. And you can again do a pack of the envelope calculation. It's going to yield Kolmogorov optimal, Kolmogorov donor optimal approximation. All right, so here is the exponent that we are going to be interested in is gamma star effective neural network of C. So remember, in dictionary approximation, we had gamma star effective of C comma D. So now the D, the dictionary, is replaced by the neural network. The neural network is now our dictionary. And therefore, we have this quantity here. And we are going to ask ourselves, is there a limit, an upper bound on this quantity? And it turns out there is. And it's the Kolmogorov exponent. So again, you see, if you pick everything carefully, and you can make it work altogether. So as to give this exponent operational significance, in the sense that it is upper bounded by the Kolmogorov exponent. And not only that, you can also show that indeed there are function classes where gamma star effective of neural network of C is equal to the Kolmogorov exponent. And not only that, if you can find a function class that you can prove it doesn't satisfy this, I would be delighted to hear about it, because we couldn't. All right. Before I go into showing you what function classes or how we identified those function classes, let me just turn things a little bit around and put things into perspective that we have shown so far. What you've seen so far is essentially answering the following question. If you have this function class and your neural network is to approximate any element in the function class to within an error of at most epsilon, the question is, how does the connectivity network have to grow at least so as for this to be possible? And it turns out that the connectivity has to grow at least as fast as epsilon to the minus 1 over gamma star as epsilon goes to 0. In fact, there is what I call a strong converse. Physicists would call a phase transition. Namely that if you exceed your gamma star of C, then you violate this error bound infinitely often as a function of the connectivity. That's not surprising, because you cannot beat the converse of covering. Good. All right. So clearly in practice, you use overparameterization and so on. I mean, so this tells you what you minimally need. And again, we don't talk about training algorithms here. OK. So let me tell you briefly how we prove now that gamma star effective neural network of C is equal to gamma star of C for different function classes. Forget about all the details that you will see in the next three slides. The main idea is the following. So the way we started this was we knew, OK, wavelengths are optimal for base of spaces. So now we somehow tried to navigate from known terrain. So we said, OK, can we approximate, wave-led dictionary is optimal with neural networks? The answer is yes. If you choose your mother wave-led so that it can be approximated well by a neural network, then what you need is how are the other elements in the wave-led dictionary generated through affine transformations and translates? Well, affine transformations and translates. Well, you have affine transformations in neural network. You just got to make sure that the neural network doesn't destroy the optimality of the approximation of the mother wave-led. And you can show that it doesn't because it exhibits an invariance property with respect to affine transformations and translations. That's it. It's all about invariances. So it's not so surprising that you can actually show for affine dictionaries, so not only wave-leds, also rich-leds, curved-leds, shear-leds, alpha-shear-leds, whatever, they are all called don't know how optimally representable by neural networks. What is more surprising is that, so this is what I call the affine group, that you can also do it for the Wald-Heisenberg group, where you have translates fine, but modulations, very different group operation. And the way we do it is always through a sandwiching argument, namely, we're going to first show what I just told you, that if you approximate this dictionary by neural network, the gamma exponent you get is at least as large as the one you would get in dictionary approximation. So you can be at least as good. And then you're going to use the Klamogorov upper bound here, and then you're going to find the optimal dictionary for your function classes, which is wave-leds for Bezov. You have gamma star of C here. And so you see that the gamma nn star effective is sandwiched between gamma star of C and gamma star of C, and therefore you have optimality. I'm not saying this is the only way to do it, but this is how we could do it. And then we went ahead and we did it for Gabor dictionaries, where you have translates and modulations. And this is, if you remember in the beginning, I said we worked hard to get this sinusoidal approximation right. This is where you need that result to get this optimality. And also for the Wald-Heisenberg group, you can show that if you start with a generator that is well-approximated by neural network, and then you translate it, and you modulate it, or shift it in time and in frequency, the optimality is preserved. And you're going to get the optimal approximation for, for example, modulation spaces as defined by Feichtinger-Gröchenig and the optimal dictionaries here of what is called Wilson basis. I understand Wilson was a solid state physicist who won an oval prize, but I'm sure some of you know this. OK, same arguments. Good. So here is the whole list. So this is what we could show. For all these pairings, optimal dictionaries for given function classes, you can do all this with neural networks with one structure. So forget about what the data structure is. You don't have to choose and find the right dictionary. The neural network will learn it for you if you have the optimal learning algorithm, which I don't. But if you do, then it's going to find it. So this one structure has a very strong universality property. It's going to find all these optimal approximations with the minimum number of bits. So you're not paying a price for this also. Again, all this granted that you know the function perfectly and you have this learning algorithm. Two function classes that are hard to approximate classically. I mentioned this in the beginning. Oscillatory textures and the Weierstrass function. Let me show you pictures. And then you see this. This is an oscillatory texture. The way it's defined is it's a cosine of some real number times the function g of x times h. So this is hard to approximate because of its oscillatory nature. And the Weierstrass function I'm sure you know about. The fractal function looks like this. And so these functions are hard to approximate. And the best known approximation results, classical approximation theory, give you an error that decays polynomially in the number of parameters you use in your approximate. It turns out, though, that deep neural networks approximate both oscillatory textures and the Weierstrass function with exponential accuracy. And this result here was recently extended by the sheet of warrant hanging to more general fractals. In fact, you can show that whenever you can generate the fractals from an iterated function system, you can be optimal with respect to neural network approximation. So to wrap up, the last item I want to show you is the case for depth. So I want to be very specific here. There's no philosophical discussions here. What I want to show you is the following. Let the FBSC3 function, satisfying some additional condition, then if you restrict your neural network and to have a finer depth, L. So let's call this shallow network because L would be small. And you restrict the network to have a width that scales no faster than polylogarithmic in 1 over epsilon, then you cannot satisfy your error bound of F minus 5 being less than epsilon. So you cannot approximate, the worst case of approximation error, is going to exceed epsilon. So the total number of parameters is going to be L times the polynomial log 1 over epsilon. And if you have a shallow network, fixed depths and the widths can scale no faster than pi log 1 over epsilon, you cannot do it. However, there is a result, the two papers that are underlying this presentation, that shows you that if you go deep and you have a finite width, namely of 23 in this case here, and again, forget about the details, the only two quantities you should look at are the depths. Depths scales polylogarithmic in 1 over epsilon. The exponent is 2 here. This is the size of the domain. The width is 23. So you have a constant times log 1 over epsilon squared, and you can get the worst case error to be less than epsilon. So here's something that shallow networks cannot do, that deep networks can do.  All right, so in summary, deep networks give you exponential approximation accuracy for a very wide range of functions. So remember, we had this algebra, x squared, multiplication, polynomials. Then we had smooth functions and oscillatory functions like sinusoids. Then they can, in principle, learn optimally vastly different function classes, so universality, one structure optimal for all these different function classes. And what is mathematically underlying this is a concurrent invariance property of deep nets with respect to time shifts, scalings, and frequency shifts. So fine group and the Weylheisenberg group, which means that you can, in principle, without knowing anything about your data, deploy this network, it's going to do as well as dictionary approximation would do if you had the optimal learning algorithm and if you have access to your function or infinite amounts of data. These are the papers underlying this, so thank you very much.


Bonds are issued at their market price, which could mean their par or face value. It could also mean at a premium if the market interest rate is less than the stated interest rate. Issuing bonds at a premium means selling them for more than their face value. Finally, it could mean at a discount if the market interest rate is greater than the stated interest rate of the bond. Issuing bonds at a discount means selling them for less than their face value. The journal entry to issue bonds at par is a debit to cash and a credit to bonds payable. A bonds payable account is a long term liability. In this example, new order issues $100,000 bonds at par, so the journal entry is a debit to cash and a credit to bonds payable for $100,000. When a bonds stated interest rate is less than the market, the bond will be sold for less than face value. We call this a discount and we use a new account, discount on bonds payable, which is a contra liability account. This means in order to use the account, discounts on bonds payable, we must be using an account, bonds payable. Also, discount on bonds payable has a normal debit balance, which is different than regular liability accounts. In this example, new order issues $100,000 of bonds at a $10,000 discount, so the journal entry is to debit cash for $90,000, debit the discount on bonds payable for $10,000, and credit bonds payable for $100,000. Actually, a better example might look like this. New order issues $100,000, 5% bonds at $96,000. The issue price is $100,000 times 96%, which equals $96,000. So, we debit cash for $96,000, we debit discounts on bond payable for $4,000, and credit bonds payable for $100,000. Then over the life of the bond, the discount balance would be reduced to zero. We call this process amortizing the discount, and you'll learn more about that in an upcoming video. Finally, let's look at the balance sheet presentation of the bonds payable after issuance. Bonds payable is $100,000, discount on bonds payable is shown as negative, but accounts don't have negative balances, so this just means that it is reducing the value of the carrying value of the bond. It also means that discounts have debit balances and debit balances reduce liabilities. The net amount is often called a bond's carrying value. Okay, when a bond's stated interest rate is greater than the market rate, the bond will be sold for more than its face value. We call this a premium, and we use a new account, premium on bonds payable, which is an adjunct liability account, or I think that's still what it's called, I haven't actually heard that term in over 20 years. Premium on bonds payable has a normal credit balance, which is a characteristic of an adjunct liability account. Okay, so let's look at an example. New order issues $100,000, 5% bonds at $108. The issue price is $100,000 times 108%, and that equals $108,000. So we debit cash for $108,000. We credit premium on bonds payable for $8,000 and credit bonds payable for $100,000. Then over the life of the bond, the premium balance would be reduced to zero. And let's look at the balance sheet presentation of the bond after issuance. Bond payable is $100,000. The premium on bond is $8,000. This adds to the bond carrying value because premiums have credit balances. Bonds, like other debts, need to be paid back. Unlike many other debts, bonds don't pay off the principal amount until the end of the bond term. So in this example, new order retires $100,000 of bonds at maturity. The journal entry is a debit to bonds payable, thus paying off the liability and removing it from the books, and a credit to cash for $100,000.


Can I take this off? Thank you all, thank you all for coming this morning. I wasn't sure how many people come to hear a speaker for breakfast, I don't often give breakfast talks. I'm a night person, so my energy level was up during the day, so we'll see how this works. I'm gonna have to figure out how to link what I do to high school football, I mean, I like that, so I'm gonna have to, I'm gonna make an assumption this morning, because I have this much longer talk, so I'm gonna skip part of it, and the assumption I'm gonna make is that everybody in this room this morning believes in free markets, that everybody in this room understands that free markets work, that free markets create a higher standard of living, that they generate more wealth, that they raise the poor, the poor are better off on a free market than any other system, in other words, that we all kind of agree on a basic level that markets create the goods, that they produce the stuff, that they create a higher standard of living, and that this country is the testament of that fact, and that history is proof of the success, the overwhelming success of capitalism and marketplace, and the marketplace, let me make one other point. When I say free market, right, I actually mean free, and free of what? Yeah, free of corrosion, free of government controls, free of government regulation, free of government controls, so I'm a bit radical in this, as you'll see, but I believe unequivocally that human history, history of the last 250 years has shown, capitalism, free markets, they actually work, they actually produce the goods, and this creates an interesting question, because if they really do work, if they really do produce the goods, if people are really better off under them, if our standard of living rises when we're free, then how come we're losing? Because we're losing, we're losing big time, and I don't care who wins any particular election because it doesn't really matter, I mean it matters, but not in the historical context, because we'd be losing not one election, not two elections, not five elections, we'd be losing for 100 years. For 100 years, this country's moving away from free markets. For 100 years, this country's been moving away from the founding principles of this country. For 100 years, it's almost exactly 100 years, you know, this year's the 100th anniversary of the two major events, the establishment of the Federal Reserve, you know, which monopolizes our money and creates bubbles and everything else, and the establishment of the income tax. So it's, you know, Woodrow Wilson is a good target, right? It's been 100 years, and I think we need to ask ourselves why, because it doesn't matter who's in the White House and who's in Congress, one thing you know, government will grow. You know, we all, you all probably love Ronald Reagan, right? And he did a lot of good things, but government spending under Ronald Reagan in the eight years he was there, doubled. Now that was good, because in the previous eight years, it had tripled, so hey, doubling is an achievement. But government grew, and of course, the Reagan Revolution wasn't really a revolution, because it didn't last. It was a Reagan blimp, right? A little period in which some respect for markets, some deregulation, lower taxes, but what did George Bush do immediately when he got out of the office? Remember, read my lips, right? Raise taxes, regulations went up. Doesn't last very long. So what is it about markets? What is it about free markets? What is it about capitalism that causes us to move away from it? And we are clearly, I mean this administration, more than any. We have been looking at Bush, right? I mean, I won't spend time here talking about too much about George Bush, but talk about government spending. So I think it's time any everybody, everybody who believes in markets, everybody who believes in capitalism, it's time we thought about the fundamental question. Why are we losing? And I'd argue, again, it's not about economics. Economics is an outside. We won that debate. Do you remember when Keynesianism was dead? Keynesianism was dead as a theory in economics. Nobody respectable believed it. Even Samuelson, who was the big Keynesian, every book, every edition of his book was less Keynesian as it came out. And yes, here we are. We live in a Keynesian world right now. Everybody believes in Keynesianism. How did it come back? It was completely discredited academically. It's the only theory that justifies growing government. It's not because somebody, they discovered the truth. All they discovered was, oh, we want more government, we want more control. So it's not about economics. We won that debate. Our people have won Nobel Prizes. We've had great economists. And again, history is on our side. The world today is on our side. As we sink in the economic freedom index, the countries that rise in the economic freedom index are doing better than we are. The freer you are, the better you do from a purely material perspective. So there's something about capitalism. There's something about capitalism that we, as a culture, resent. That we, as a culture, object to. That we, as a culture, reject, turn our backs on. And I think unless we understand what it is, we cannot reverse this trend. We can keep teaching it economics and keep teaching economics, but that's not what it is about. It's not about economics. Because again, the fact, history, theory, everything is supportive markets, and yet we move against it. The problem is much more fundamental. The problem is that we live in a culture that resents capitalism, and it resents it because capitalism is about what? What is capitalism about? What are markets about? What are people going to the marketplace to engage in? Well, if you're on the producer side, if you're making stuff like Apple, what's Apple in the business of? Profit, they're in the business of making money. You guys know it, but it's a little embarrassing to say it out loud. It is in the culture we live in. The profit margins in this are huge. Steve Jobs did not wake up every morning thinking how to make my life a better life. He woke up every morning thinking how he could make a cool device that he could make a lot of money on. He did, and he does. And when I went to buy my first iPhone, 2008, economy spiraling out of control, I went to buy it because I wanted to stimulate the economy. Ha ha. That's why you guys go to the mall, you go shopping because you care about your fellow man and you wanna make sure that jobs out there in the economy and you go buy stuff for the sake of the, right? You buy stuff in the mall because you care about whom? You. You. Capitalism is a system in which people go out, you see this in the marketplace, any kind of marketplace, primitive and sophisticated.  On the producer side, it's the fun of making a product, it's the beauty of making a product, it's the money that's involved in making a product. On the consumption side, it's making my life better, cooler, more productive, whatever it happens to be, but it's about me and for the producer, it's about him. And this is not a new observation. This is not an observation I made or I ran made, this is Adam Smith, wealth of nations, 1776. He says when the baker makes the bread, he's not thinking about you, he doesn't care about you, he's making the bread because that's how he makes a living for himself. That's how he takes care of himself, that's how he feeds his family. And when the grocery owner sells you the bread, he's not doing it for you, he's doing it for himself. Capitalism is built on the principle of self-interest. It's about self-interest and I know that makes us uncomfortable. Shouldn't, but it does as a culture. I'll give you a simple example. When Bill Gates makes $60 billion for himself by doing what? What is that, how does he make $60 billion by selling us a product that we want, willing to pay let's say a hundred bucks for because it's worth how much to us if we pay a hundred bucks for it. Now it's worth more, otherwise we wouldn't pay a hundred and it turns out it's worth much more in terms of the networking effects, in terms of the benefits of our lives. We generate a lot more than $100 of utility for the $100 we pay for Bill Gates's product. So he makes $60 billion. What is the culture's view of Bill Gates as he's making $60 billion by making all of our lives better?  That he's just a greedy businessman. So from a moral, ethical perspective, how do we view him as a culture? At best, neutrally, because he's making a lot of money, that's some virtuous, and at worst he's a crook and let's sick the Justice Department on him so they can break him apart, right? When does Bill Gates become a good guy? When he starts giving it away. So notice, making money, creating wealth, building something, impacting everybody in the world. I mean, there's not a human being on the planet who has not been impacted positively by Microsoft. That morally neutral at best, negative at worst. Giving the money he made over there away, impacting a much smaller group of people, having very little impact on the world as a whole, that is the essence of virtue. Now he's a good guy, wow. How would he become a saint? If he gave it all the way, moved there to a tent and could show some peace. Because what for us is virtue, right? Being self-interested, that's our virtue. When we were this big, we were taught that's advice. Virtue is what? Being selfless, sacrificing, giving, sharing, that's virtue. But those are not the virtues of capitalism. It's not what drives people to go to work every day. It's not what wealth is created from. Wealth is created from, as Adam Smith said, from people self-interested. And yet that's advice. Our ethical code that we live with every day is inconsistent with the economic system that we advocate for. And you can see that. There are a lot of successful businessmen in this room, I'm sure. I attended a Lifetime Achievement Award ceremony in Charleston, South Carolina a couple of years ago for business leaders. And they handed out these awards and they read long bios, because this was a celebratory event. And they read these bios and this is Lifetime Achievement for Businessmen, right? 90% of the bio was about community service and charity, and 10% was about their business achievement. Where do you think you really have an impact in the world? When you're charity and community service, and I have nothing against charity and community service. I've won a non-for-profit. But really, let's think about this. Where did vocates have more impact on the world? It make building Microsoft and its charitable activities. Building Microsoft. Most of you have had more impact on the world through your business than you do through your charity and your philanthropy. Yet, that's what gets talked about. The charity and philanthropy is what gets talked about. That's what gets emphasized. That's what makes you noble and good? Well, I'm not convinced of that. What makes you noble and good is building something, creating something, creating wealth, creating jobs, creating a business, creating a product, making stuff. That's what this country's about. Charity, community service, they're fine. But then the 10%, the 90% is the building, is the creating. You know, this country has an amazing history. We started in 1776 as a third-rate British college. I mean, we weren't that important because the British, if they'd taken the war seriously, they might have won. But they had the French and the Spanish to worry about.  In 1776, within 140 years, 140 years, we were the mightiest economic power on the planet. We had to enter World War I just to get it settled. Now, whether we should have or not is a different question, but we were stronger than any other country. The British Empire was in decline and we were at the top. How did that happen? What happened during those 140 years? We built something. We created. It was, business happened during those 140 years. The Carnegie's, the Melons, the Rockefellers, the J.P. Morgan's happened during those 140 years. A lot of wealth creation happened during those 140 years. It did not happen because we were free because we provided them with the protection of property rights and we created a free society in which they could thrive. But it's business that built this country. It's business that created this country. And what do we do today? We call them robber barons. And we're not comfortable defending them, but they are the real heroes of 19th century America that built this country and created what it is. And yet we denounced them and we demeaned them in our public schools and our universities. We're uncomfortable with them. If we think about Carnegie, what do we think of? The libraries. We think of libraries. We don't think of steel. But Carnegie changed the world. He changed the world we live in, not because of the libraries. I mean, the libraries again are nice, they're great, wonderful, but what counts is the steel. He built an industry in this country. He made us the most significant industrial country in the world. That's what we should be remembering him for, but we feel embarrassed about it because he didn't do it for the common good, which is what the libraries are supposed to be there for. He did it for Carnegie. He made a lot of money on it. It's my belief that as long as we hold self-interest as this kind of discomfort was a negative from an ethical moral perspective, we will fail. As long as we hold that our moral obligation and moral nobility, that morality is all about being selfless, the morality is all about the other. Morality's not about ourselves, we will lose. That is a morality consistent with socialism. Selflessness is a consistent with socialism. It's not consistent with capitalism. And you can see it in every political debate out there. Somebody over there is suffering and has a need. Well, it's your moral obligation to help them. That's what we've been raised on. So all the government is doing is stepping in and helping you become a better person. And most of us will vote for it. So there's a group over there they need, they need some healthcare. They didn't have it, so we're gonna give them a little bit of, you know, we'll create Medicaid to help them or Medicaid. And then of course there's a second group over there that now that we've taken care of those guys, we haven't taken care of the other group and we need to help them a little bit and we'd pull at your heartstrings and good. It's your moral responsibility to help them. And they need stuff and you're not helping them.  So all I'm doing as a politician is helping you, again, be a little bit more moral, a little bit better person to assist that other group. And you start with a small program that just helps a small little group and soon enough 50% of all healthcare is being provided by the government. And we haven't stopped there because with Obamacare it's much more than 50%. Because there's always another group that needs and another group that needs. And as long as it's your moral obligation to provide for their needs, that's it, you're done, that's socialism. The entitlement state, the whole entitlement state is founded on this principle that it's your moral obligation to help others and you're not doing a good enough job at it. You're keeping too much of your wealth for yourself so we, the Neverland government, are gonna help you achieve that goal and of course we all vote for it. You know, how many of the richest counties in the United States voted for Obama? Of the 10 richest, how many voted for Obama? Eight. And the reason I voted and why, right? They all voted for tax increase on themselves. That's exactly what they did. Because they knew the first thing he was gonna do was advocate for taxing the rich. It wasn't a secret and he had eight counties out of 10 but it's not, you can say, oh, it's cronyism. They expect my Silicon Valley cronyism now. They're not getting any money from the federal government yet they voted for it. In California, we just voted for a huge tax increase, 30% tax increase on the wealthy. How do you think the wealthy voted? For taxing themselves. People do not vote their pocketbook. This is what the Republican Party doesn't understand. People don't vote economics. People vote what they think is right, what they think is just, what they think is fair. That's what Obama gets and Democrats get and Republicans don't. They have them all high ground they're left as. We never challenged them on them all high ground and they went and we vote for them. And it's all this talk. In California it was like, if you don't vote to increase your taxes, we're gonna have to raise tuition on these poor kids and they need an education.  Okay, we know it's our more responsibility, we'll do it. And today we have the largest, we're facing the largest redistribution of wealth in human history. It's called Medicaid, which nobody challenges. Every dollar that you put in, some of you might be on Medico already, every dollar you put in has been spent, 0.1, right? You're gonna consume $4 of every dollar you put in, which means our kids and grandkids are gonna have to pay four times the taxes that you did in order to pay for your health care. That's morality for you, right? I mean, how does that work? But you need the health care and it's okay for them to sacrifice for you. That's the logic, right? So as long as we hold that logic, it doesn't work. Now there's another aspect of this kind of mild code that I wanna quickly bring up and that's the regulatory state. We have two aspects of statism. Entitements and regulations. I would argue regulations function also based on this mild code. What is that perception of self-interest? When I say businessmen all about self-interest, what immediately pops to mind? What have we been taught since we were this little about self-interest? Self-interest leads to what kind of behavior? It leads to people doing what? Greed, which means lying, cheating and stealing, right? Right, so people with self-interest that are lying, cheating, SOBs. And that's what businessmen know. That's the perception out there. They're lying, cheating, SOBs.  You don't make money that way. But the culture believes that deep down because self-interest equals lying, cheating, SOB. Businessmen are self-interested. Therefore, simple logic, simple deductive logic. Therefore, all businessmen are SOBs. And what do you do if you think people are gonna lie, cheat, steal from you? You regulate them. You control them. Every elevator you walk into in the elevator, here I'm sure there's a little diploma on the wall that says that a government bureaucrat is inspected the elevator and won't fall. Because we know, the left of their own devices, elevator manufacturers will build elevators and they kill people. So we need a government agency to tell us that the elevator's safe because the market couldn't take care of that, right? That's nonsense. But we don't trust. We don't trust the market. We fear the market because it's all about self-interest. So we regulate. This became very clear to me. Remember Enron, WorldCom, those guys? Right? They caught like five guys committing real fraud at five big companies in the US. And ultimately the guys went to jail. But what was the outcry in America when that happened? They were all crooks. And we need to catch them all. And the way to do that was a bill, right? What passed as a consequence of Enron? Saubanes, Oxley. Public custody's economy of $1.5 trillion has caught zero crooks, didn't prevent the financial crisis, just a big burden on American business. Past the Senate, remember how any conservatives voted against this bill? 98 to zero, it passed the Senate. It's disaster of ability. Talk to people in sizable businesses. It's destroyed the IPO market in the United States.  And it basically places a little bureaucrat on every CEO's shoulder monitoring every decision being made within the company because we can't trust them. I was on Bill O'Reilly in 2002. Bill O'Reilly, just after Enron and so on, Bill O'Reilly wanted every CEO in America fired. And I, you know, I think the clip is up on YouTube somewhere, but I did defend CEOs and said, you know, in this country, you're still innocent until proven guilty. He got so angry, I've never seen Bill O'Reilly so angry. He turned red, he was pounding the table. They all have to go, they were all crooks. It comes from the same place. I don't trust them because they're all after money. They're all greedy somebody said. But in the worst sense of the word, we don't trust them. So my view, this is Rand's real contribution. I am Rand's real contribution to debate about capitalism. It's the question, the maw code that we all take for granted. It's the question, the idea, the nobility virtue come from serving others. That serving others is the essence of morality. She rejects that. And she presents an alternative, a morality of self-interest. A morality that says no, self-interest properly understood, morally understood, it's not about lying, cheating, stealing, and being an SOP. It's about building and creating. It's about taking care of yourself. It's about making your life the best life that it can be. It's about living life to the fullest. It's about being the best human being you can be. It's about using your mind, because that is the tool we use in order to survive as human beings. The values we have around are your products of our reasoning mind, of our rational mind. So she challenges the morality that exists in the culture. And she says unless we're willing to do that, we will lose this battle. This is what Aval Shrugged, which many of you, I think, have read. This is what Aval Shrugged is really about. It's not about economics. It's not about politics. It's about morality. It's about ethics. It's about what is truly noble and good. The heroes and activists are noble and good because they create, because they build, because they make stuff, because they take care of their own lives.  They truck, right? They go on strike. But we need to challenge the moral code that pervades the culture. And in a fundamental sense, and I know there are lots of questions. I mean, this is pretty, if you have questions about what that means, I'd be happy to answer them. But in a deep respect, this is a return to what this country is really about and what this country was really founded upon. You know, I think the most important political document in human history is the Declaration of Independence. I think it's more important than the Constitution because I think it sets the context for the Constitution. And the Declaration of Independence does not, it assumes a certain revolution in man's thinking. There's something unique about this country that is not true of any other country in human history. This country is founded on a moral ethical principle. It's founded on a brand new idea. I mean, this revolution wasn't about teetaxes. It wasn't about any particular action of the British. It was about a principle. And the principle is the answer to the question of, who does your life belong to? Because before the founding of this country, the answer was simple. Your life belonged to whom? The king in the case of England. The tribe in the case of a tribe. You know, whatever the dictator of the time was, it belonged to somebody else. Every society going back to the beginning of time, your life as an individual was meaningless. You were always a cog in some collective machine. The founders of this country said, no, my life is mine. Your life is yours. We are ends in ourselves as individuals. The founding documents of this country don't articulate your commitment to society. They don't articulate you as your brother's keeper. They do the opposite. They articulate your responsibility to yourself. You have an inalienable right. Inalienable means nobody can take it away. To what? To do what? To maximize social utility? To help your feather man? Did you charity? No. You have an inalienable right to your life. And we've lost the concept of right. The left has stolen that concept from us. They distorted it and they converted it. But right means a freedom of action. It means you are free to pursue your life. You are free to live your life as you see fit.  What soda drinks to drink, for example. It's your life, yours. Not societies, not the neighbors, not the governments, not the kings, yours. That's a profound statement. And it's really the first time in human history they said it in a political document. It was stated in a political document. So you have a right to your own life. Inalienable, nobody can take it away.  And it doesn't matter because it's inalienable. And you have a right to liberty. Your liberty. Which means have the thoughts that you wanna think and execute and act on those thoughts. And as long as you're not hooting other people, as long as you're not violating other people's rights, you should be left alone. That's what it means. And in the most self-interested political statement in human history, self-interested political statement in human history, each one of us has an inalienable way to pursue our own happiness. Now that's, again, profound. And it's not what the left means. It's now we have to give them healthcare and food and jobs and men and weight. Now it means we need to leave them alone. It needs exact opposite of what the left says. What rights mean is that you have a right to be left alone. You have a right to follow your dreams. You have a right to pursue your own freedom. But those are all individualistic concepts. This is all about you pursuing your life. This is all about self-interest. In a deep sense, this is all about what it takes to establish a free capitalist society, which we had when we took those ideas seriously. So if we wanna win, if we wanna bring back free markets and capitalism and the success that is America, then now these are the principles we have to resurrect. We have to reignite the spirit of a right to life, liberty, and the pursuit of happiness. And if we do that, we win. Thank you all. Thank you.