Greetings from the National Archives. I'm David Ferriero, archivist of the United States, and it's my pleasure to welcome you to today's virtual book talk with Michael Bulel, author of Inventing Equality, Reconstructing the Constitution in the Aftermath of the Civil War. Before we begin, I'd like to tell you about two upcoming programs you can view on our YouTube channel. On Wednesday, January 27th at noon, you can hear Deborah Willis discuss the Black Civil War soldier, a visual history of conflict and citizenship, a kaleidoscopic yet intimate portrait of the African-American experience from the beginning of the Civil War to 1900. The book shows how photography helped construct a national vision of blackness, war, and bondage, while unearthing the hidden histories of those black Civil War soldiers. And on Friday, January 29th at noon, Benjamin R. Justison will discuss his book, Forgotten Legacy, William McKinley, George Henry White, and the Struggle for Black Equality. Historians have long overlooked President William McKinley's cooperation with prominent African-American leaders, including George Henry White, the nation's only black congressman between 1897 and 1901. Because of McKinley's dedication to the advancement of African-Americans and the safeguarding of their rights as U.S. citizens, he might be considered the first civil rights president. The rotunda of the National Archives building enshrines the Constitution of the United States, but the four pages exhibited there are the beginning of this guiding document of our national government. Shortly after its ratification in 1788, the 10 amendments we know as the Bill of Rights were passed, and over the next 200 years, we have appended 17 more amendments. In inventing equality, Michael Builiel looks closely at three of those amendments, the 13th, 14th, and 15th, passed after the Civil War to extend freedom and rights of citizenship, to formerly enslaved persons. As the records of our nation show, ratification of these amendments was not the end of our debate. Our voluminous court records preserved cases brought before the United States courts, even up to the Supreme Court seeking equal treatment under the law. Using these records, Builiel brings us an examination of the evolution of the battle of true equality in America. Michael Builiel is the author of numerous books including Revolutionary Outlaws, Arming America, 1877 America's Year of Living Violently, and the People's History of the US Military. He's taught at every educational level in the United States, Germany, Italy, and United Kingdom. Awarded the Bancroft Award in 2001, he is the only person to ever win both of the Article of the Year awards from the Organization of American Historians. As the founding director of the Interdisciplinary Violence Studies Program at Emory University, Builiel worked with community organizations and public agencies to study and respond to the numerous effects of violence in modern life. He's also organized special educational programs to aid veterans of the Vietnam War and served as educational director of the Veterans Institute of New England. Now let's hear from Michael Builiel. Thank you for joining us today. Good day. I am honored by this invitation to speak at the National Archives about the core principle of human equality and this day after the inauguration of our second Catholic president and first woman vice president. We as a nation pride ourselves as living in a land of equality and opportunity, the two being intricately and inexorably linked. It is also obvious that the United States has a long history of struggling to make the ideal a reality. Given the political language of this country which tends to promote the concept of equality, we must ask why are we still fighting to ensure, for instance, that black lives matter? So let's go back to the start. The two American founding documents each made clear the core problem for the United States. Declaration of Independence of 1776, you surely know these words. We hold these truths to be self-evident that all men are created equal, that they endowed by their creator with certain unalienable rights that among these are life, liberty and a pursuit of happiness. Thomas Jefferson provided a perfect expression of human equality. But what did Jefferson mean by self-evident? Was equality self-evident, not requiring any evidence? Sadly, not even Jefferson accepted equality. One of the most bitter and instructive ironies in American history is the fact that the same man wrote these eloquent words for claiming equality and just four years later, the first book Justifying Slavery on Racial Grounds, notes on the state of Virginia in 1780. Jefferson can never get his head fully around what he meant by equality, struggling with the concept for the remainder of his long life. He also is responsible for the one perfect metaphor of racial attitudes in the United States, his relationship with Sally Hemings, the half-sister of his wife Martha. Over the 75 years after Jefferson wrote the Declaration, you will be hard put to find a single document supportive of equality. What you will find instead is a steadily increasing number of books, articles, speeches, and sermons supportive of inequality, which nearly every political, intellectual, and religious leader insisted was natural. They would deliver stirring July 4th orations, praising the United States as the land of equality and then turn around and find all sorts of exceptions. But let us go back for a moment to that other founding document, the Constitution of 1787. When I was young, we had to memorize the preamble, we, the people of the United States, in order to form a more perfect union, establish justice, ensure domestic tranquility, provide for the common defense, promote the general welfare, and secure the blessings of liberty to ourselves and our posterity to ordain and establish this Constitution for the United States of America. Again, the wording seems so clear. Our Constitution is proclaimed in the name of the people, not of the states or just white Protestant property-owning men, but we're the people. The Constitution does not say, never bothering to define citizenship. Congress was supposed to decide, but the first Congress left it up to the states, which generally just sort of went with those born or naturalized within their borders as citizens with, again, many exceptions. Immediately, we get all sorts of contradictions. What happens when a state refuses to acknowledge that members of a specific religious or ethnic group are not worthy of citizenship? And what of that half the population, which is not male? The Constitution may say people, but every state read that as men. Why? Whenever a court bothered to address the issue, they usually dismissed the concept of women being equal in the eyes of the law as a ridiculous violation of reality. For in their view, women did not have the physical or mental capacities necessary for participation in public life. Such a proposal also violated the traditional legal concept of Coverture. The American understanding of the legal status of women came from that renowned British legal scholar, William Blackstone, who defined Coverture in 1765, to quote, by marriage, the husband and wife are one person in law. That is, the very being or legal existence of the woman is suspended during marriage, or at least is incorporated and consolidated to that of the husband under whose wing, protection and cover, she performs everything. This lack of legal identity was most obvious in her loss of her name upon marriage. Women did not have rights, but duties. A female was to be obedient first to her father and then to her husband. A woman without a man was a relic. That was the legal phrase. A leftover, with few exceptions, women could not own nor acquire property, write a will or enter a contract. They had no legal identity, but were utterly dependent on a male. There was clearly no self-evident truth of equality when it came to women. Not even all men were included among those eligible for the legal equality of citizenship, but different times and places, various religious and ethnic groups have been excluded from the equal protection of the law. One of these exclusions brought on the most violent event in American history, the Civil War. Obviously, I'm speaking of slavery and its supportive ideology of white supremacy. Slave holders from Thomas Jefferson to Jefferson Davis did not hesitate to label the United States the land of freedom while insisting on the inferiority of those they felt justified in enslaving. The big problem for the United States was that ever more states rejected slavery. And though still guilty of racist policies, these states insisted on freedom for their citizens of African heritage. So what happens when a black citizen of a free state travels to a slave state or a slave finds his way into a free state? It is important to note that racism warped every aspect of American law, politics, society, culture, and language. For instance, given the tendency of white slave owners to rape their female slaves leading to lighter skinned children, it often became difficult to define blackness. And what happened when black sailors for the United States Navy was integrated stepped ashore in a slave state. In 1822, South Carolina declared that they would be sold into slavery. The Supreme Court declared their law in constitutional South Carolina did not care. Southern states opened mail checking for anti-slavery sentiments and terminated all First Amendment rights. There was no education for blacks or for most whites for that matter because there was no public education. Through the 1850s, these contradictions began to tear the country apart. As those living outside the slave states became more dissatisfied with the efforts of the slave owners to nationalize their institutions. Occasionally, one of the Southern leaders like John C. Calhoun would make a claim for the supremacy of states rights over national rights. But even he wanted the federal government to expand its powers in order to hunt and recapture runaway slaves. As divisions intensified Chief Justice Roger Taney that he could solve the contradictions by terminating the concept of equality. In 1857, the Supreme Court issued the Dred Scott decision stating that a black person could never be a citizen since the Constitution was written by and for white men alone and that black people were property by birth. Instantly, most people in free states saw the problem. There would soon be no free states as slavery would be nationalized. As Illinois Senate candidate Abraham Lincoln best summarized the situation and I quote, a house divided against itself cannot stand. Either the opponents of slavery will arrest the further spread of it and place it where the public mind shall rest in the belief that it is in the course of ultimate extinction or its advocates will push it forward till it shall become lawful in all the states north as well as south end quote. Lincoln captured well the fear that the Dred Scott decision would lead directly to the creation of a slain republic. That certainty led to Lincoln's victory in the 1860 election, the secession of 11 slave states and the Civil War. Now a strange thing happened in the Civil War. There was little doubt that most white Americans were racists in 1860 in the sense that at the very least they were white supremacists. This was true of Lincoln as well who repeatedly stayed that he felt white men superior to blacks. However, Lincoln turned back to Jefferson's understanding of equality to reframe the issue. He argued that Jefferson had put forth a simple understanding of equality that entitled everyone to the natural rights of life, liberty and a pursuit of happiness. Lincoln thought it reasonable and right to let black Americans decide for themselves how to live. But the declaration had put forth the radical notion that each individual is best able to determine his or her own happiness. Most northern whites seem to have shared that simple and minimal understanding of equality at the start of the Civil War. There was a general feeling that everybody deserved the right to decide for themselves how they would live given what they perceived as natural limitations of age, ability, race and gender. But as the war progressed, white people from the free states began meeting blacks and discovered that they were people too. In fact, admirable people. And I'm gonna give just one example. One of the greatest joys of conducting historical research is prying in reading the private letters and journals of unknown people, gaining access to the mental process of common Americans. For instance, Chauncey Cook, a farm boy from Wisconsin, had never seen a black person in his life until he marched down the Mississippi with the Union Army in 1863. He was delighted with them, writing in his family that they were the most gentle, nicest and polite people he had ever met. They were also deeply grateful to the white soldiers for fighting to free them. Those positive characteristics stood out all the more as, quote, every white man and woman in Mississippi is ready to shoot a poisonous, end quote. The Union troops learned that, quote, the Negroes were our only Southern friends, and let's not forget they were Southerners. And they kept us posted on what the whites were doing and saying, bringing us food along with military intelligence. As he marched through the South, Cook became convinced that slavery had damaged the whites almost as much as the blacks. The elite had deliberately kept poor whites ignorant of the outside world, denying them equality as well. By his count, only one-tenth of the Confederate prisoners could read and write, account matched by most studies. By the time the war ended in 1865, most Northern whites and a great many Southern whites felt that black Americans deserve citizenship. Similarly, black and white women had repeatedly proven themselves brave and committed citizens as thousands came forth to serve the cause of freedom in a number of ways, as nurses and spies, office clerks and guides, support staff and recruiters for the military. In fact, the person praised by Lincoln and Frederick Douglass as the most effective exponent of the Union cause was Anna Dickinson, the first woman to address Congress in a courageous orator who stood down pro-slavery mobs. In the same way, immigrants and Catholics left no doubt that they too willingly risked everything for their country. By the time the war ended in 1865, a consensus had spread through the country that those previously denied the rights of citizenship had proven their worth and deserved full legal equality. The result was a trio of amendments that sought to correct two glaring contradictions in the Constitution. It's unstated support for slavery and its failure to define citizenship. The first issue was resolved by the 13th Amendment, which ended slavery in the United States. Defining citizenship proved a little bit more difficult and contentious, resulting in the 14th Amendment, which became part of the Constitution in 1868. Its opening words appeared to make clear the legal equality of all Americans in eloquent and precise language, and is worth quoting, all citizens, excuse me, all persons born or naturalized in the United States and subject to the jurisdiction thereof are citizens of the United States and the state wherein they reside. No state shall make or enforce any law which shall abridge the privileges or immunities of citizens of the United States, nor shall any state deprive any person of life, liberty, or property without due process of law, nor deny to any person within its jurisdiction the equal protection of the laws. Nothing can be clearer. A literal reading of the opening paragraph leaves no doubt that all persons are equal in the eyes of the law. However, as is so often the case in American history, the eloquent evocation of rights is soon followed by the qualifiers. The second section of the amendment states that the vote may not be denied to male inhabitants. Women were outraged. Leaders of the women's rights movement had agreed to put ending slavery first with the understanding that their legal equality would then be addressed. But the all white male Congress undermined that understanding. As Elizabeth Cady Stanton said, the insertion of the word male and the first time gender had been inserted into the constitution into the 14th amendment undermined the concept of women's equality and set back the women's suffrage movement by a century. Congress followed up with the 15th amendment which protected the right of black men to vote. Very specific again in the gender. The initial result of these amendments was the period known as reconstruction and the nation's first true, almost democratic elections. The Republican Party swept elections around the country including in the South where alliances between former slaves and poor whites led to dramatic changes such as the first public schools, infrastructure, projects, and the creation of a free press. Former Confederates hated it all and they counterattacked often quite literally through violence and intimidation that Ku Klux Klan and other white supremacist groups determined to force inequality in the nation and it worked. They found a powerful ally in the Supreme Court which slowly whittled away the meaning of the reconstruction amendments. When the brilliant Myra Colby Bradwell applied for the Illinois bar, she was rejected because women could not be lawyers. Bradwell appealed to the Supreme Court citing the 14th amendment as establishing equal rights. In 1873, the Supreme Court dismissed her suit on the basis of nature and divine ordinance. Nature had made women timid and delicate and thus unfit for civil life to quote the decision. God had made women to have one job and one job only. Motherhood. For these reasons quote, a woman had no legal existence separate from her husband and could not hold property or enter into contracts and therefore cannot practice law. All human laws, even the Constitution the Supreme Court declared must bend to the law of the creator making any notion of equality between men and women silly and downright dangerous. Two years later, Virginia Minor appeared before the court using the 14th amendment to insist on her right to vote. Again, a unanimous court rejected Minor's argument. Her status as a citizen was irrelevant. Entering into perfectly circular logic, the court reasoned that since women are citizens but cannot vote, that women did not have a right to vote. When Congress attempted to protect the equality of black Americans with a series of civil rights acts, the court overturned them on the basis of congressional overreach in trying to enforce the 14th amendment Congress had violated the 10th extending its reach into private matters. If a Southern state wanted, if Southern state governments wanted to limit the rights of their black citizens, that was their business. The court impatiently noted that 18 years had passed since the Civil War, surely it was time to move on from these issues. The Civil War and Reconstruction taught white supremacists that they had to keep whites and blacks apart or they might come to see one another as equals. The white supremacists Southern state governments therefore began a policy of segregation in the 1880s, the so-called Jim Crow laws named after a common stage stereotype of blacks. They recognized that stereotypes only work if the races do not get to know one another. Inequality also gained powerful intellectual support in 1877 when a new concept appeared in the United States, social Darwinism, perceived as a result of indisputable science, social Darwinism became an obsession among educated Americans. Even ministers who as a group rejected Darwinism celebrated social Darwinism, the idea that the successful deserved their power while any effort to aid the weak undermines natural selection and thus human progress. Social Darwinism also rated the races with the whites conveniently on top. While rather self-serving for the successful and the white, it also cleared the consciences of the majority of white Americans. Social Darwinism was a clarion call for inaction. Thus ended America's brief experiment with equality and it returned to inequality on a national scale.  For instance, in 1882, Congress passed its first national immigration legislation, the Chinese Exclusion Act, barring all future Chinese immigration. This act was not repealed until 1965. In 1893, the Supreme Court issued their decision Plessy versus Ferguson, which rejected the notion that segregation violated the 14th Amendment. They admitted that the amendment sought to enforce the absolute equality of the two races before the law, but in the nature of things, it could not have been intended to abolish distinctions based on color. The court could not imagine how anyone thought that a law establishing, quote, merely a legal distinction between the white and colored races could in any way undermine legal equality. A distinction is not discrimination, separate but equal facilities do not imply inferiority. No evidence was required to prove that segregation hurts no one since it simply, quote, is too clear for argument. As Justice John Marshall Harlan pointed out in his stunning dissent, the Supreme Court chose to ignore reality. They ignored the violence, the paralyzing effects of racism on both the victims in American democracy, that separate is never equal and the fact that the Constitution is colored blind. There was a more subtle, yet equally devastating form of inequality, exclusion. Pretending that women, blacks, Chinese, Hispanics, Jews, at all, were never part of American history is a deeply harmful form of inequality. For a century after the Civil War, historians ignored these groups and wrote almost entirely about white men. For instance, 270,000 black troops fought for the union during the Civil War. I just finished a search of dozens of history textbooks published between 1880 and 1950 and cannot find a single one that mentions this essential fact. My own high school textbook, The Growth of the American Republic, written by the prominent historians, Sandro Morrison and Henry Cominger, had a two-page section out of 830 pages on slavery that began, and I am quoting it, as for Sambo, whose wrongs moved the abolitionists to wrath and tears, there was some reason to believe that they suffered less than any other class in the South from its peculiar institution." This absurdity is followed by assurances that the slaves were happy, sang, and danced, and learned a lot from their masters who never used violence. We were supposed to believe that nonsense. You don't have to. What we now see clearly is that, in a single decade of the 1860s, thousands of Americans discovered human equality and invented a clear understanding of legal equality that has formed the foundation for every struggle for positive change since. Progressive forces never stopped fighting for equality, and in the years after World War II, they found the nation once more willing to believe that all persons born or naturalized in the United States are citizens. It's a beautiful concept, and one that still inspires the advocates of equality. Thank you.


So, this video concludes the Armored Course series. It was a lot of fun for me to share what I know with everyone about my favorite game, and as of today, I can rest knowing that I have passed on what I know to the next generation of Ravens, which in many ways is ironic, because that's actually one of the main themes of the game itself. Back then, when I was younger, I spent countless of hours after school, fiddling around with my AC in the garage and exploring every corner of the game and trying out new combinations of parts and weapons, and obviously as an adult, you know, with life and responsibilities now, I don't really have the same opportunities to do that anymore, but it really makes me smile to see so many new players having that same experience that I loved as a kid happening today for a fresh new generation of AC players. Right after this video, I have a few housekeeping videos coming up to let you guys know the state of the channel going forward, and soon after that, we will be in our usual game design and computer art related content. Real quick though, for those of you who are new to the channel and subscribed for Armored Core content, don't worry, I will still be playing Armored Core 6 on my break time, you'll probably see it as my default background noise when I'm talking about things on live streams, and speaking of streams, yes, I do plan to make a more regular schedule with live streams starting next month, I just haven't had the time to figure out what the best day in time to do that would be yet. I'm thinking two days a week for one to two hours each, we'll probably have one day dedicated to game design and community stuff, and one day just dedicated to Armored Core, where I just grab a handful of people on a team, and we just play against whoever shows up online. If you are a founder member on YouTube or Patreon, then I have added a new perk that allows you once a month for an hour, you can team up with me and fight together against random online opponents in Armored Core 6. And if the connection is good, we might actually stream the session so everyone else can enjoy the fun with us. Assuming that you're okay with that, of course. Obviously, as you can probably tell, I really love playing teams in Armored Core, that's where I think the game really shines, especially if you have good communication and coordination with your team while you're playing. So if that sounds like fun for you, you can also ask questions and just talk about whatever you want during this session. This perk really is just a fun new way for us to hang out and enjoy some games together. The reason why I'm limiting this perk to the founder tier is because I simply do not have a lot of free time. But if you're supporting the channel on that level, then I feel like it's the least that I could do to show how much I appreciate you. So that is an option for those of you who are interested. Either way, with this series complete, as an old Raven, I feel I have done my duty to pass the torch. And just as the operator said in the game, I've given my wings to you now. Where you fly and how far you go with them is up to you. So go now. Let's see just what kind of future is born from your decision. You have the right and the duty to find that out. And as always, hope you have a fantastic day and I'll see you around.


Thank you You ready? I'm ready when you are. Let's do it Okay Action today, I'm going to be watching every film that is nominated for best picture at the Oscars 2023 this video has no affiliation with the actual Oscars Society company dynasty this video is so you guys can get a sense of the Oscars this year and you can sound a little bit more knowledgeable about the films at your Oscar watch parties on Sunday if you guys want me to do your dirty work and watch all the movies So you don't have to and recommends which ones you should or shouldn't watch you guys should subscribe I took a deep dive into my back end and I saw that a bunch of you that are watching are not subscribed Well the perks of subscribing you get to be here You get to watch my videos if you turn on the notifications you can be here right when I am every single time I post I am responding and interacting with comments because I love seeing what you guys have to say so be the first one to hop on my videos to hop on my back end and Conversate with me in the comment section down below I'm gonna be watching all the films and I'm gonna be taking you through my thoughts about each film in depth I have a rubric system this time So it's a little bit more structured so you guys can differentiate my personal opinions Versus the actual plot acting and those opinions We have five things we're gonna be testing these movies on today The first one is plot plots are super important to movies obviously They are what we pitch movies as when we are talking to our friends Usually we go into the plot summary weren't talk about a movie not necessarily the acting the score and these other elements Usually those are more afterthoughts. So the plot is going to be a very important thing that we're gonna be looking at whether it's entertaining Whether it's interesting moving sad Whether it's unoriginal We're going to be testing it to its limits next is gonna be pace When I say pace I am talking about the speed in which the plot moves forward One thing the Oscars gets criticized for is that almost every single nominated film is a snooze fest meaning that it is Slow it is boring and it is something that you would fall asleep to it's something all Procedure films get criticized for and I am not one to shy away from that I very much love fast-paced movies, but I am not going to let that bias come into play when I judge the pace of these Movies it is not going to be if it's slow and I don't like slow movies personally No, it's going to be if that slowness or if that pace that is set throughout the movie is worth it Next is gonna be score. I love movie scores. I listen to them in my free time I think they are so crucial to any movie as we all know this one is kind of just in there for giggles I personally love judging scores if some I love it when a score sticks out to me So we're gonna see which score stand out and which ones fall into the shadows That doesn't really mean anything coming from me because my knowledge on music is even less than my knowledge on film So why are we doing this video? Next is gonna be the acting acting is literally all the Oscars is about Most of the time Oscar season is just about the performances from the actors It's a snooze fest for each performer and actor a part of the Academy, which is fine. I love it I eat it up. I think it's great I love praising actors and deciding who was better and who was worse I love pinning people against each other and last but not least for this rubric system. This is the Trin factor This is basically just if I personally enjoyed the movie if I thought it was fun and fresh if I would go see it again Because at the end of the day, it's my world. Everyone else is just living in it I can create whatever reality I want to and I'm going to be a part of this rubric system. I can control the narrative of this video You and I'm going to make it just the way I want it Trin factor. Does Trin level like it? Let's go and here are the nominees for best picture I Everything everywhere all at once is the only movie of these that I saw in the year 2022 the other ones I did not Top Gun Maverick Bella sleep during triangle of sadness. I got Distracted Elvis. I tried to turn it on twice Got distracted and left the room today. We are going to be watching every single Today, we are going to be watching every single movie nominated for best picture Let's get it. Let's get it today's first watch is the fablemen's We started off this watch journey a little shaky being as the power went out in the middle of this movie because there was a Tornado all as well me and my dog are safe The movie is directed by Steven Spielberg and stars Paul Dano Michelle Williams and Seth Rogen this movie follows Steven Spielberg and his introduction to film and the journey It took to become a picture maker It also follows the challenges he faced and his family dynamic while growing up This plot was really interesting to me and it was really amazing that Steven Spielberg got to make this movie about his own life I don't think anyone else could have told his story better Obvi this movie has some serious undertones But it also has this slice of life energy, which is very Charming in my opinion with many troubling subjects as your parents having a fair bullying at school Antisemitism Steven Spielberg still threads a long comedy through the movie Which is done in a really seamless way that makes the movie so charming in my opinion I gave this plot an 8 out of 10. I thought it was a really wonderful story to see I think this movie has a true love for film behind it so you get to see that through the eyes of the lens and It really does give a sense of nostalgia even though I never grew up during that time It's packed with this childlike innocence growing up very much a coming-of-age Inspiring story to see someone like Steven Spielberg and see everything that he went through this movie has such a care and love for cinema that is so Wonderful and Exciting to see we haven't seen a movie really love cinema this much in a while And I was really in love with how it was depicted the acting in this film was a top tier Michelle Williams was Incredible she was in your face the entire movie and she just kept coming back with this drama this hysteria this This raw realness that was just like watching a train wreck, which I loved I loved seeing her character go through all this disarray But still have this like element of comedy behind her and I think that Michelle Williams did that very well Which I think it's like a hard role to play this kind of chaotic mother who has like mental problems who's having an affair but also has this like underlying comedic role to her which I thought was very interesting to see I think Gabriel Lebel did an amazing job I am so surprised he didn't get an Oscar nomination. He was so phenomenal I think that his ending dialogue scene between him and his high school bully was Literally so good that would have been like his Oscar nom video that they played and I wish he got nominated for best actor because He really deserved it. I think he did an amazing job throughout this movie And I was just in love with his performance. The score wasn't anything too special in my opinion I can't really remember anything that stood out to me But like I said earlier in the video that doesn't mean much coming from me being as I don't know much about scores So I gave the score a five out of ten. I just really wanted something to stand out to me That's really what this score rubric is for to see if a score really stood out to me And made me want to like download it afterwards the pace I gave a seven out of ten I think it was worth the slight lulls throughout the movie there's a little bit of a lull throughout the first act and Maybe like two or three times. I felt a little bit impatient with the pacing of it And that really goes to show my attention span not really a reflection of the movie I think the slight lulls through the first act paid off at the end I think it was ending worth the wait all in all I think this might be one of the saddest yet hopeful movies I've ever watched one of my favorite quotes from the movie was You don't owe your life to anyone not even me stood out to me loved the delivery I gave this movie a four stars. I just thought it was super charming and I Would see it becoming best picture in another year I think this year has a bit too many strong competitors for fablemen's to really shine But I wish it did this is triangle of sadness. This movie is directed by Ruben Otzland Sorry if I didn't pronounce that correctly the movie stars charbly dean creek harris dickinson woody harrelson and dolly dylan this is a movie all about wealth and capitalism and communism It is a Shit show of a movie being as the plot is just shoving you in the face with satire on society But I quite liked it the movie follows a couple of influencers who are invited to a luxury cruise ship alongside a group of Out of touch wealthy people this all takes a very unexpected turn when a brutal storm hits the ship I actually surprisingly liked this film I tried watching it like a couple months ago and I ended up turning it off because I was just kind of not paying attention But after sitting down and watching it it was so funny. I laughed out loud while watching it It also has so many great dialogue pieces And at the end of the movie it really depicts this social hierarchy Turned once they end up on the island. I think the contrast between the first act and the last act was Wonderful to see I loved how purposeful it was through the first act of the plot I was a little bit lost on the direction that they were going through But they have a few Key hooks that keep you entertained and keep you drawn into what's going on the second act was a little bit messy But in the best way possible it has a lot of chaotic scenes a lot of like Out of context scenes that you think are out of context but kind of all fall into place once the third act happens I thought the third act was lively and funny Creating a great dynamic between all these characters It has great twists and additions to the relationships the characters have with each other on the island for acting dolly d freaking Killed it She was like a sleeper character that came up in the last act and just freaking annihilated everyone When she came on screen in that third act, she commanded your attention Just like she did with the characters and it was so brilliant to see Best performance ever for the pace. I would give it probably a Seven out of ten. I think any slow parts paid off in the end. I think everything was very purposeful in this movie Every bit of satire was placed very purposely within the mix of drama I thought the score was like a seven out of ten. I wouldn't download anything from it It was it didn't stand out too much to me But each scene was elevated by the score and in a way that made sense So i'm not saying that it distracted me, but I nothing stood out to me to download it The trend factor is that it is trend approved. I loved the final act. I thought it was a really entertaining movie I don't think I would watch it too many times after this. I would definitely watch it with a friend This was definitely in my top watches of the best picture nominees I think my favorite quote from the movie was I think it's unsexy to talk about money And I love the title revealed that the triangle of sadness Represented the triangle of sadness between your eyebrows when you're scrunching all in all I think I would give this movie a four out of five stars Elvis is directed by Baz lerman and stars Austin butler and tom hanks. You guys are gonna hate me for this one I watched Elvis and I could not get through it I have tried watching Elvis two times before I sat down for this video and right when I turned on this movie to film I got hit with the biggest wave of depression induced by my period I was So upset while watching this movie Watching this movie made my pain 10 times worse I don't want to watch this movie I don't want to have tried to watch this movie and I just can't Get intrigued by it and I feel really bad because people are saying this is great movie And I cannot for the life of me get involved with this plot and care about what's going on I lasted way longer than I did before I think I got into like an hour and a half of the movie before I had to call it quits because first of all I was in a lot of pain and second of all I was dying because I just didn't find it interesting I think austin butler does an amazing job, but tom hanks just kind of ruins all the scenes for me I really don't like the plot narrative to choose to have the story told through the manager's point of view I think I would have much preferred it just being through Elvis's point of view But at the end of the day, I do not care about Elvis and this movie did not make me care about him either I think the pace is fine But the length of the movie is so long that it doesn't really matter your pace because it is such a long movie The score was fine The trend factor I would not watch again It was very hard for me to sit through being as the length of it and also other factors being as Mother nature ripping out my insides my favorite quote from this movie was heat's white I would give this movie a two and a half out of five stars being as I didn't give it a fair chance But also I didn't like what I saw of it This movie follows the women of an isolated religious colony when a shocking secret about the colony's men is revealed For years in this colony, the men have occasionally drugged the women and then raped them The truth comes out and then the women talk about their new situation and what they will decide next This is based on a true story. It's a very heavy drama It's a very heavy drama This is based on a true story. It's a very heavy drama and it is Breathtaking this movie made me hold my breath while watching it. The plot is the plot is difficult to watch it is difficult to hear about it is Heartbreaking and it is also empowering while watching the women have to decide between three things fighting and staying surrendering and leaving Or staying and surrendering to the men's request to forgive their rapist I gave the plot a 10 out of 10 the pace was actually fast with immediately telling the story It's heavy on dialogue But each scene is so important in moving the plot along quickly This might be my favorite score so far behind everything everywhere all at once The score was daunting and still hopeful at times with also this somber blanket cast along all of it I know that makes no sense But it was definitely one of the most moving scores that i've watched so far the trend factor I really liked this. I don't think I would watch it again. That's the only Downside of this movie is that seeing it once was enough for me I think this would be a really awesome class film if I saw this in class I would really enjoy it But I overall really enjoyed this movie and there was a lot of quotes and Dialogue pieces that really stuck with me Some of my favorite quotes from the movie was this story ends before you were even born An act of wild female imagination and without language for it There was a gaping silence acting was 10 out of 10 Every single woman put their whole heart into this movie I mean and and we weren't not and we were not expecting anything less from this stellar cast I mean, I knew they were going to go crazy in this movie with their acting abilities So overall a four and a half out of five stars for women talking Top gun maverick Top gun maverick did not stand a fighting chance against me being as I hate tom cruise with a burning passion The first time I tried watching this movie over christmas break I fell asleep and my parents were eating up the movie. I've also never seen the first one so Any hits of trying to get me to reminisce on the first movie were lost The plot follows tom cruises character after more than 30 years of service as one of the navy's top aviators Pete maverick mitchell is where he belongs His character is now a teacher and he is pushing the envelope as a courageous test pilot and dodging the advancement in rank that would ground him Maverick must confront the ghost of his past and his deepest fears to then go on a mission that demands The ultimate sacrifice from those who choose to fly it I know a lot of people like this movie and I just could not get behind it. The score was actually pretty epic I thought the score was actually probably one of the best parts of the movie I think the acting was really good from what I could tell it wasn't anything crazy I wouldn't give anyone a nomination over it. It was very much just there Um, like I said, I have a huge bias against this movie. So I would ask someone else about this movie I would not take anything of what I'm saying with any real validation for how you feel about the movie because I know a lot of people like this movie. I just Couldn't it was the same as with Elvis like I know people like it. I just could not get behind it There was something that I just This one I knew why this one. It was literally because of tom cruise I just don't like tom cruise and seeing him on screen Invokes rage in me. I don't like watching his movies. There's not a there's not a single tom cruise movie that I enjoy watching Kind of a long movie I think it's it's two hours and 11 minutes and it definitely feels it I told you I was going to be unbiased with this rubric system and I am not I am sorry The trend factor for this movie this movie goes hard for some people and I'm just not one of those people know when for the trend factor, but If it wins your factor, then that's all that matters Maybe you want to know all my thoughts on everything everywhere all at once I have an entire video dedicated to it. So I'm not going to waste time here talking about how much I love the movie It's a five out of five for me. I think it was amazing But the link will be in the description and right on the screen right now if you guys want to go check it out Tar directed by Todd field starring Kate Blanchett This movie falls renowned musician Lydia tar and the inevitable downfall of her career This movie is definitely not for people who don't like prestigious art I think this movie has a very specific theme It wants to follow and it dives headfirst into it It takes the obsessed artist and puts you right through the lens of that character A lot of the times when we see obsessed artist movies We are looking at it from a third point of view, which can be great But it also takes this element of the are away from it We're realizing how prestigious and kind of silly being so addicted to your craft can be whereas in tar It takes it all very seriously the plot is your typical downfall of the obsessed artist seeing someone how Seeing someone who is either trying to achieve everything or has everything and inevitably landing at rock bottom by the end of the film This movie is really heavy on dialogue. So if you're not a fan of that, I would not suggest this to you Personally, I have been enjoying those films lately being as with women talking and this movie They're both very heavy on dialogue and everything is written within the pages. The acting is phenomenal Kate is undeniably captivating in this role. There is No other person that could play it. She literally becomes the character. I forgot that she was an actor I forgot that Lydia tar was not a real person while watching this film And I think that is a wonderful place to be in. I have to give it to Kate I would not be surprised if she snatched that best actress because she was just undeniably amazing I think she was so fantastic. There are so many difficult one shots very long pieces of dialogue that she just Nails i'm obsessed with it. I love her and I love her in this role I think this is one of her like Best roles ever and I would not be disappointed if she got best actress The acting was a 10 out of 10 along with her counterparts. I think everyone in this movie was really good But this movie was k-blanchets and she just knocked it out of the park. The score was amazing But of course it had to be amazing. I was I was not expecting anything less than the score to be amazing Of course 10 out of 10 for the score Of course downloading it all now. It is Thrilling it is exciting. It makes you excited about classical music and I love that. I think that is An amazing thing to have within movies to get me excited about something that I have practically no interest in I think the pace has a little bit of issues within the third to second act. I think the pacing I would give it like a Like a six out of 10. I think it has a constant beat throughout the first Two acts and then that transition between the second to third act There's like this lull within the rhythm of the story and I think that kind of drew me out a little bit So I wish they kind of kept that constant beat that was going on through the first and second act the trend factor I loved it. I This one had my personal bias being as that I love obsessed artist movies I love the downfall. It just checks all of its boxes. This movie Was lit. This movie was very lit and I would recommend it to everyone There's a lot of good quotes from this movie. But my favorite quote was U-Haul lesbian tar gets a three and a half out of five stars for me being as I don't think the Watchability is great for me personally that might change with time But me with my attention span right now. I would not watch it again, but I would recommend it to people So it gets about a three and a half to four stars for the movie avatar I could not watch it because it's not available to rent and it's not in theaters. So I guess I just missed it So avatar you get a three out of five just because I couldn't give you a fair shot. So Pity stars, but I'm sure it's great. I just Cannot watch it because it's not available. I would go to the theater and see it, but I just can't It's just not in any theaters near me All quiet on the western front is a war movie directed by edward burger It stars daniel brule and felix camera This plot follows the war that breaks out in germany in 1914 Falls paul and his classmates who quickly enlist in the army to serve their fatherland And very soon after being drafted. They realize the horrors of what actually is taking place I'm gonna be very brief about this. I don't like war movies. They are my least favorite movies ever There has not been a single war movie that I love Not a one I just despise them I want to take my review on this movie too seriously being as that I already have a bias against it because I don't like war movies But I will say this movie wasn't bad. The score was one of my favorite parts They have a single score that kind of plays throughout the movie It keeps coming back almost like kind of like a motif But it's a really like interesting score to have within a war movie for some reason it felt very like daunting but still a little bit more like bright Then a typical somber war score If that makes any sense, please let me know because I feel like that doesn't make any sense The plot and the concept of this movie was actually really interesting for a world war one story I think that a lot of times like I said if you've seen one more movie you've seen them all But this movie was actually pretty interesting in the way that it chose to show The character's point of view You really sink down deep into the brainwashing of these characters and the true shock and horror that they face Whenever they end up seeing the tragedies that is actually going on They stage some of the first act as if it's almost a coming of age. It's very light. They are very giddy and Excited to do this. This is the this is their honor. This is this is like going on a school field trip for them It's very much framed as a coming of age and throughout the war scenes And the battle that they're in there's a bunch of teen distortion throughout it Of course, this movie is going to be really hard to watch, but it is worth a watch I think it was actually one of the best war movies I've watched Particularly because I don't really like the genre and I actually found myself liking this movie Like a good amount like for someone who has no interest in war movies I think I like actually like this movie a fair amount.  I thought he was in the game for a while. I Always find war movies too slow. This one wasn't too bad. I think it picked up I think it had a nice beat throughout the movie. So I don't have much to complain about I would give this movie a three out of five stars just because Of my own personal bias But I I want to give it a high enough score because I don't believe it's like a bad movie I just wouldn't watch it again because of my own Personal opinions. I have so little to say about The Banshee of Anishiren because this was me when I watched the movie I am so sorry. There was no time in my filming to re-film and re-watch The Banshee of Anishiren I have to start editing this video. I'm so sorry Colin. I love you Colin Like I really do but this movie literally put me to sleep And I had tried to watch this movie two months ago And I fell asleep then too So I'm so sorry Colin, but like it was a no for me. I snoozed I'm coming to you from my laptop while editing this video But that was the completed oscars watch for this year's best picture nominees This was a very trend level video. You cannot get any more trend level than this I told you I was going to give you opinions about the films I told you that I was going to be unbiased and I wasn't I wasn't at all Sorry, I'm sorry I should be better about that, but there's tons of reviews out there showing you a completely unbiased review very analytical of every film We don't need another one, but those were my thoughts on this year's best picture nominees I think that everything everywhere all at once is going to snag best picture I think it's probably between everything everywhere all at once and maybe in my opinion It would be between everything everywhere all at once and the fable men's It just don't see the fable men's having that great of a chance I'm literally thinking about the list of films. I really don't have one that's Coming up there by everything everywhere all at once personally my ranking of the oscar nominated films putting aside the banshee of an isherin and avatar because I clearly didn't give them a fair shot I would put Elvis at the bottom Then top gun maverick then all quiet on the western front Women talking triangle of sadness the fable men's and then everything everywhere all at once and you can say that I'm wrong, but I'm not in charge of who wins. So it doesn't really matter what I think That was it for today guys. I hope you guys liked it It was a little bit of a different format than what I usually do, but I enjoyed it So let me know if you guys enjoyed it. Uh, happy oscars. Have fun at your oscar watch parties I will be In my apartment all by myself watching the oscars like I always do Yes, I'll see you guys later. Thanks for watching. Bye


Hi everyone, welcome to Petabyte Scale Logging with FluentD and FluentBit, a use case from Intuit. My name is Anarok, I'm part of the company Calyptia and I help run product care. So who are we? So today it's going to be myself and Hansel who's joining me from Intuit, who's on the cloud observability team there as a senior software engineer. And for myself I'm part of the OSS group that is a maintainer for the FluentBit project and I help drive product for both FluentBit and FluentD. So for those who are unfamiliar with what is FluentD and what is FluentBit, they are cloud native computing foundation projects. They are part of the graduated projects alongside Kubernetes, Prometheus and others. They started 10 years ago and the primary problem they solved is how do you take data from point A and send it to point B. And if we look at this problem over the last 10 years where you see that data sources continually evolve, we have more things like mobile applications, system logs, Kubernetes, microservices, containers, app metrics and we have additional destinations that just keep compounding. You might have things like Amazon S3, you might have Splunk, Elasticsearch, Loki, Azure, GCP, Kafka and those destinations just continue to evolve and expand. You might need to send data to multiple, you might need to send data to all of them, you might need to send data to a single one, but in any case, FluentD and FluentBit are really vendor neutral solutions to saying let's collect data once and send it to as many destinations as you require. Now when we look at challenges for logging at scale, we have to look at this in perspective of a broad overview. One is high scale can really equal very high costs. You can have things like reliability and buffering, how do you make sure that logs get sent where they're supposed to and you don't want to open up a request every single time that you have a new message. Think about that when you're sending 10,000 to 100,000 messages per second. Networking, always an issue here with things like ephemeral workloads and Kubernetes or you're in the cloud, you might have air gapped environments, how do you make sure networking doesn't hurt you when you're doing logging at scale. Event throughput, you might need to maximize how many messages per second you're sending, you might have latency requirements, you might have to get messages so that your developers, your operations folks can go and start to debug and diagnose all that information. Security, so again in a large scale environment we might have sensitive information that might make its way through how do you look at securing the data transition and then last but not least when you're running this in production how do you look at operationality or ways to minimize the performance impacts at a larger scale. So what we'll talk about in this session is how into it solve many of these challenges but also how D-Influent Bit have solved these challenges over the course of the 10 years that that project's been running. One is filtering parsing and compression, so support for compressing workloads, being able to take data and throw away stuff that isn't used, parsing data so you can take log messages and make really good sense out of them so taking like for example an Apache HEP server log and giving you the additional information like IPs which you can then go and enrich with geo information things like source IP, the request type. Reliability and buffering, so within the D-Influent Bit there are file system and memory buffers so if you need really fast performance and you want to send both messages you can use the memory buffer by default but both of these projects also support file system buffering which is great in case of things crashing for example you might have a server that goes down but because you're storing everything on the file system you're able to recover all of that information none of that data is lost they all follow and at least once mechanism which leads into networking so when networking goes down we don't lose data we have the configurable retry mechanism as well as back pressure handling so as the load increases from say a file or an application Fluent Bit is able to understand that maybe the application can't receive too many messages at the second and we'll retry at a desirable time in the future and especially when you're logging at scale you might have thousands and thousands of endpoints or thousands and thousands of applications logging data how do you make sure that all these retries don't try exactly at the same time and unfortunately cause a denial of service these are places where we've built a lot of that functionality within the open source Fluent D-Influent Bit Event throughput, another place where if you need additional performance and you have the resources to dedicate you can use aspects like multi-worker configuration in both Fluent D and Fluent Bit and give you the ability to utilize a lot of the server resources that you have at your disposal with security there's two places to this if you're securing sensitive information you can use anonymization filters you can use parsing to remove those secure fields and then of course with TLS both Fluent D-Influent Bit support the ability for encryption and transit operationally we can use things like different architectures so both products or both projects have been designed with this ease of use and flexibility in mind you might want to minimize the resources spent say at your source within containers within your Kubernetes nodes within your servers and do more of that processing at a centralized layer and we call that the forwarder aggregator called architecture so just some common architecture patterns right we have the forwarder only which allows you to take all of these sources and independent pieces and send data they each individually handle back pressure so you're not necessarily relying on a single point to grab all the data enrich it and then send it out the disadvantages is you have to do a little bit more configuration management and it is sometimes hard to end more destinations which falls a little bit with the configuration right if I'm sending data to Splunk, Classic Search and Kafka all at the same time I might have to go and configure that individually on a forwarder basis now a common architecture pattern we see with large scale is forwarders with aggregators so that is going to be less resource utilization on the edge devices all they're doing is collecting that data and forwarding it on to an aggregator it allows you to process and scale the aggregator to your independent you can add more back ends in a single tier and then of course the disadvantage here is you do have to dedicate some of the aggregator instance now that's all for just a general overview of how Fluent-D and Fluent-Bit are projects are placed within the Cloud Native Computing Foundation and now how you can look at some of the challenges that they solve for at scale I'm really happy to pass it over to Hansel who's going to be able to talk a bit about the use case specifics add into it and give us an idea how petabyte scale can be achieved in reality so with that let me hand it over to Hansel thank you entire now we'll get into the use case add into it into it provides financial solution for consumers, small businesses and self-employed individuals major products from into it include TurboTax, CubeBooks and Mint add into it Kubernetes is adopted widely across four business units it's used extensively at over 30 segments with more than 5000 engineers onboarded to it we also have over 80 engineers developing our Kubernetes platform we'll see how Fluent-D and Fluent-Bit are used to handle logging from Kubernetes clusters add into it the use case is to transport logs generated within the container which is returned to STD Out or STD Everstream to our persistent logs store to enable this we have a daemon set deployed in the cluster this ensures that a logging part is present in each and every node this part contains Fluent-D or Fluent-Bit which is then responsible for collecting and forwarding the logs from that entire node the local logging plugin fights logs into the files which are tailed and processed the log events must also be enriched with some metadata this would enable us to identify the cluster, namespace, port and container where each event originates from when running search queries at the log store and all this has to be fast and efficient we want to be able to transport huge volumes of data with as low end-to-end latency as possible we have a centralized persistent log store where all customers can see and query log events we wanted a common path to transport all the logs and supply plan this would simplify the connectivity between the VPCs containing the Kubernetes clusters and the VPC containing the log store you only have to expose the log store to the pipeline instead of all the clusters this helps us avoid sending the log events over the internet sending data over the internet is very expensive and we can cut out the cost drastically by sending data through AWS's network only during any connectivity issues with the log store once the buffers get filled up, the logs are gone for good if you have an intermediate store for these logs, this can be prevented this also provides an opportunity for other applications to read the raw log data this is especially useful for security, compliance, and analytics applications this is our Naive approach, which is a streaming pipeline there is a Demonset deployer in the cluster, so each Kubernetes node will have a logging pod running in it this pod will tail the log file from the host, Mounted as volume there is a FluentBit plugin that gets the Kubernetes metadata from its API and enriches the log events with that these enriched log events are then sent to a Kinesis data stream the events are then routed to Kinesis Firehose now the Kinesis Firehose has a native capability to write to the log store and that's how the enriched log data gets to the persistent store this provides a reliable scalable mechanism to transport the logs from hundreds of Kubernetes cluster onto a centralized persistent log store we only need access to the log store from Firehose and this data transfer happened within AWS's network also the logs can be read from Kinesis data stream while just using a consumer this pipeline was working well for us initially but as we started supporting more and larger cluster we started facing some challenges with this pipeline we had a hard limit on the number of events that can be transported every second in each pod with the solution that used FluentD as a logging container we were capped at 2500 events per second per node once we switched over to Fluent with base logging container we were able to double that limit to 5000 events per second per node however this was still not enough especially for larger nodes which contain a lot of containers packed inside also as you saw there are two hops between the cluster and the log store each of these hops were introducing some latency which was quickly adding up we started seeing the median end to end latency getting as high as 30 seconds this was not acceptable for our most time critical applications so we decided to reduce the number of hops and hence the time it takes for the transport to maintain this streaming pipeline we had to have some persistent resources running the log load that needed to be transported was extremely elastic especially based on different times of the day however these resources even with the auto scaling enable were not able to scale efficiently to match that log load this was making the pipeline inefficient in terms of the cost and ended up being too expensive we wanted to cut down some costs as well so in simpler terms the target for the new pipeline were to increase the throughput dramatically without increasing the resource consumption by that poor reduce the end to end latency of the log transport pipeline as much as possible and to minimize the cost needed to maintain it and this is where the S3 pipeline comes into play this is how the architecture of S3 pipeline looks like the space enclosed by the blue dashed line represent a node running in the Kubernetes cluster each node contains a daemon set pod containing fluently the logs written by the container on its STD out and STD out streams will be written as files that are rotated in the node and it's all done by the Docker daemon these files are exposed to the container by using a volume mounted directly onto the host the fluently container then tails these files and listens for any new log events and for any new log events fluently pushes it onto a buffer which is specific for each container at a certain fixed interval 10 seconds in this case the buffers are flushed onto an S3 bucket object after compressing it with gsit so each S3 object would contain logs generated by the container for 10 seconds compressed as a gsit file in the pipeline fluently is not responsible for enriching the log data with metadata but rather writes the logs as is onto the S3 object now we have a separate deployment in the cluster which ensures that a single pod will be running across this cluster this deployment communicates with Kubernetes API and gets information about any new containers for any new containers created in the cluster it will fetch its placement and other metadata including its labels and annotations and pushes it onto a separate S3 bucket within the log pipeline VPC we have a deployment running in AWS Fargate this deployment listens to any new objects in the log S3 bucket for any new object in this bucket the path would indicate the container for which the logs are for the log pipeline deployment then fetch the corresponding metadata from the metadata bucket both these are then sent over together onto the log store this communication happens over AWS's transit gateway the log store is then able to apply the metadata to each and every event in the log S3 object the key takeaways in the architecture are that the log data is compressed from the cluster and only deflated at the log store and all the data transfer happens within the AWS network and moreover process and transfer happen via a transit gateway created specifically for that purpose as you have not seen Fluendy is deployed in the cluster as a demon set this ensures that there is a single part in every node and each part is then responsible for transporting logs from all the containers in that node regardless of how big that node is from this we can realize that this part would become the bottleneck and the weakest link in the chain so the throughput will be set by the immune that can be transported by a single part the biggest gain in the throughput can be achieved by making the demon set process transport more and more events one mechanism to achieve this is to do as little processing as possible at the demon set part that is the key to note in the new pipeline design next we wanted to identify all the things that can be offloaded from the Fluendy and how we facilitate that function elsewhere after multiple iteration we were able to pick two CPU intensive processors that can be offloaded without any change in the events at the log store these were providing us with this massive challenge throughput this is the first thing that we were able to offload by timestamp parsing timestamp has to be parsed from each event for a couple of things to attribute a timestamp for each event and to identify the starting line of new event in case of multi-line events this was very CPU intensive but we were able to offload this onto our log store without any change in the end user experience the second thing we offloaded is the metadata enrichment in the previous pipeline each event was enriched with the required metadata by Fluendy itself this has now been offloaded and done by the metadata deployment external log pipeline deployment and the log store the metadata deployment communicates with the Kubernetes API to get the required container metadata in an S3 object the external log pipeline deployment then collates this big metadata with each S3 log object which is also container specific both these are then sent together to the log store which can then apply the metadata to all the events in the file the additional advantage with this is that the metadata enrichment happens in batch across all the events in an S3 object at once this reduces the processing needed and the amount of data transferred when looking at the cost breakdown for the pipeline we found that the data transfer cost was the highest component higher than even compute and storage so if we can reduce the amount of data sent over and use a more cost effective way to transfer it we can make huge savings in the cost one added benefit is that as we reduce the amount of data transferred the latency also goes down as mentioned before Fluendy compresses the data via gzip in the buffer when it's flushed to an S3 object normal log load we were getting around 10 times compression this gzip file then be returned to log store as is what that means is that the log data stays compressed as soon as it leaves the clusters and until it reaches the log store or the log data is always compressed in transit just this one change cut down the data transferred to one tenth the volume to that of before also as we are applying the metadata in batch for an entire S3 object even need not have the metadata along with it this was also giving us large savings in terms of the amount of data transferred by cutting down the size of each pro log event this is when we compare it to the previous string pipeline we also have set up an aws transit gateway between our log pipeline vpc and the log store vpc and all the data transfer happens through this this was giving us around 60% in cost saving as opposed to sending the data over the internet using an advocate way to demonstrate how this new pipeline performs I have set up a test when it is cluster with 3 broken nodes each node contains the demenset part and a few other parts to generate some sample log events this cluster also contains the metadata deployment as you can see from our metrics dashboard that the 3 nodes it's able to transport 90,000 events per second without any errors so that means that each node can support up to 30,000 events per second we also see that the CPU of the part stays stable and that it can constantly support the log over a period of time the log generator in this each node collectively was generating 30,000 events per second and we have 3 such nodes so the total log load was around 90,000 events per second and I ran this log test for an hour so that should give us 324 million events at our log store that's exactly what we see in our log dashboard we can now see that the end-to-end latency which is a time difference between the container logging an event and the log store indexing it we see that the median end-to-end latency is just around 9, 8 seconds and the 99th percentile is less than 14 seconds that's a massive drop from our streaming pipeline let me give a summary on the improvements of this S3 based pipeline over a streaming pipeline that we saw from the demo in the case of throughput we saw a massive 6 times jump in the number of events that can be transported from 5000 events per second per node to 30,000 events per second per node moreover we have just check transporting over 1 gigabyte per second from a single Kubernetes cluster next up we see that the median end-to-end latency drops to less than 1 third from around 30 seconds in the streaming pipeline to just over 8 seconds in the S3 based pipeline and the latency drops considerably when considering the 99th percentile which has the latency cut down by more than 75% finally looking at the cost we see the cost saving of more than 92% when compared to the streaming pipeline this is due to not having to keep position resources running compression of the log data and using transit gateway this was coming around to more than 50,000 US dollars saved over every petabyte transferred and was able to create a petabyte scale logging pipeline at Intuit using Fluent D and Fluent Bit. Thank you all so much for participating in this session and I would like to open up the forum for questions at this point. Thank you.


What's going on guys got Kyle here with latent sports cards, and we are ripping 22 23 tops Chrome you wave a museum collection 12 box full case number six that are randomized for the break guys I'll show you again real quick One second There we go Florent be with a life-sick and Celtic down to Craig and with Copenhagen Marseille. Good luck guys Yeah, it's uh, yeah, it happens man It's that that OBS when it sometimes when you hit the little intro button We have sometimes it just crashes the whole system for whatever reason man It has happened quite a few times with us No big deal though. We'll keep on moving guys I'm ready. Good luck guys tops museum box one. We are back. We are back Sorry about that guys Rookie is Pablo Torre start us off Barcelona Saudi Amane base Rookie of Julian Alvarez and 2125 Raphinha for Barcelona Nice one here for a beat or yeah Dortmund 225 got Kareem Adiyemi on the auto Nice one there Dortmund on the low numbered Kareem And we've got Jersey Otto to 299 and Gonzalo Ramos An extra jersey Otto here Gonzalo Ramos for Benefica and we've got Guardiol or Vardial On the dual jersey Otto there Now when it's numbered to 299 for Leipzig And Jersey on the back is 275 Virgil van Dijk for Liverpool on the triple relic And that's when they're Liverpool box two Yeah, guys be sure to hit that like button again Appreciate if you liked it the first time but uh We did lose the stream there Appreciate you guys coming on back Harry Kane base Luis Diaz Rookie is Etiquette Zlatan and nice one here to 50 Levent Dalski for Barça We've got Otto to 299 Harvey Elliott archival autos for the for Liverpool Liverpool definitely seems to have a lot of cards in this set no doubt Harvey Elliott and right behind it. We've got for Liverpool Trent to 199 On the jersey Otto nice one there Trent Alexander Arnold And this one is for Leipzig to 250 Dominic soba sly Meaningful material. He's on Liverpool now Soba sly there for Leipzig box three Base is Darwin Nunez Pedri is Holland Messi 299 we have Benjamin Sesco And Otto 275 this is Paul vonner for Bayern Paul vonner Otto And another Virgil van Dijk this time to 199 Liverpool again on the jersey Otto VVD to 199 And some 99 on the back we got Rogerie meaningful material for Man City Rogerie box four Thomas Moeller Mason Mount Base Chua Mani got 2125 Ryan Gravenbirch and a rookie to 75 Enzo Fernandez It's Enzo And 250 we've got archival autographs gold that is Anzhu Fati We've got Jersey Otto to 75 Kai Havertz I believe he's on Arsenal now to 75 Kai And Jersey this one is to 150 Pedro Cancalves Known as for sporting box five There's Luca Modric Hey, there you go Chris that'll work We'll get we'll get you some Arsenal fans by way of Chelsea or some Arsenal players Foden, Gravenbirch, Sesco and rookie to 99 we got Pablo Torre And rookie is going to be 299 rookie reference Otto of Isak Johansson That's got to be Copenhagen on the rookie Otto there for Craig to 299 Oh nice one here to 50 we've got a patch Otto Thomas Moeller Nice one there Byron the legend Thomas Moeller 26 to 50 it's Michael with Byron nice uh nice Moeller there And rookie Jersey to 99 we got Fabio Carvalho for Liverpool Definitely some Liverpool in these boxes man Does seem like it from the couple cases we've done Got a Pierre Emric Aubameyang base Got Kevin De Bruyne Gérard a rookie of Simmich And caca number to 75 AC Milan We've got Otto to 299 archival autos for Liverpool It's going to be Darwin Nunez And dual Jersey Otto this one is red Number to 25 it's going to be Diego Jota also for Liverpool With the two sick patches there match used on that memorabilia Nice Diego Jota there to 25 Liverpool It'll be Wednesday morning Michael Forest will run it Wednesday morning when he gets in There's the 50 Christopher and Kumku for Leipzig there Meaningful material I think he's on Chelsea now Bunch of movement went around that last transfer window But yeah Michael I think Wednesday morning is when we're going to do it We've been keeping a tab the whole time so Forest will do it on Wednesday Either before the stream starts or right after he does the breaks I believe it'll be before though Frank Lampart Matisse Telbase Ian Rush We've got Benzema and to 50 it's Trent Alexander-Arnold This one is Jermaine Defoe to 299 archival autographs for Tottenham there Little throwback Otto Jermaine Defoe Jersey Otto to 99 is Sandro Tonali AC Milan nice young Italian And De Bruyne there number to 250 is going to be the blue for Man City Nice KDB one of the best midfielders in the EPL KDB Alrighty guys got five left Well this one's heavy That's got to be the gold bar or the gold frame Sure is sure is Got Franz Beckenbauer Pulisic Jamie Gittins rookie Harvey Elliott and to 99 we got Messi Nice one there PSG on the number Messi Here you go Christopher We've got to 99 on the gold frame Otto It's going to be Frank Lampard for Chelsea Nice one there on the Frank Lampard gold bar or a gold frame black ink Otto Here you go Chris to 99 I will mag that one for you Chris Nice Lampard there That's sick Very nice We've got Gerson to 199 That's going to be for Olympic On the Jersey Otto Momentous material and 250 We've got Ryan Gravenberge for Ajax meaningful material Yeah that's a sick one Chris That is sick Lampard was a beast for a long time man I think he's a coach now right Somewhere Got Nico Borea Susich got the 125 Luca Mojure Nice one there Real And to 25 Nice low number one here Jude Bellingham That's going to BVB the newest Real Madrid midfielder Nice Bellingham there to 25 And another one for Chris to 99 is Giorginio On the archival autographs Chelsea Chelsea they gave him the black pens I guess Giorginio and Lampard got the black ink Otto We've got to 299 This one is Jack Greilich Man City on the Jersey Otto Nice Greilish there to 299 Keishim with the Man City spot And Mason Mount This one will be to 75 for Chelsea as well There you go Chris It is Mason Mount meaningful material Player worn on that memorabilia to 75 Three left Hey appreciate you getting in Star Wars Thank you thank you We have some baseball tomorrow morning with Matt I think there should be some spots left If you check the break schedule you can usually see What breaks are going down and at what time I do appreciate you though grabbing some Star Wars Get that going here at the end of the night Got a base Del Piero Jeff Felix Musiala Roy Charlson And to 25 Nice one there for Liverpool Gerrard Stephen Gerrard Got Fira Hashi On the archival autographs To 299 That one is for Celtic Fira Hashi Nice one here Tottenham Number to 50 We've got Hungman Sohn On the Jersey Otto Sick one there Tottenham That is player worn on that Jersey Very nice Sohn there 29 out of 50 Nice hit Tottenham That's Brian Very nice Sohn there Brian And to 75 We've got Kai Havertz Momentus Material Another one for Chelsea And 2 left Yeah that's a sick one man To 50 on the Sohn that'll do He sells really well man he's got a big fan base Good luck, good luck 2 left Florian Vertz Yeah sounds like you've got to get the enemy more often Chris It's working out It's working out Jude Bellingham Vinny To 125 is Ansu Fati And to 75 Roberto Carlos To 99 we've got Georena That's going to Dortmund On the 99 Otto And Jalph Felix the 299 This one is for Athletico Nice one there Jalph Felix And Pedri on the back is number to 50 Jersey Momentus Material Jersey Nice one there for Barcelona Nice young player Pedri And last box here Good luck guys this was Museum Collection Case number six Yeah that that does happen Definitely socks are popular Yankees We do schedule like I was saying earlier We do schedule a couple days ahead of time So if you guys are ever looking for a specific Like break or a release day or something A specific team I would definitely check a few days before that Break is scheduled Or before release day or something like that We always schedule a few days before So you can find your team a little easier maybe Now there's no exact time on when he schedules But it's typically in the mornings So if you check a couple days before a break Or before a release day You can usually find your team There's Ian Rush to 125 And to 99 Harvey Elliott Ooh nice one here to 75 for Real We got a Luca Modric Very nice one there Archival autographs for Real Nice one there Eric Luca Modric one of the best midfielders in the world And Chris again Number to 50 We got Jersey Auto Mason Mount That's a nice one there Three out of 50 On the Player 1 memorabilia Mason Mount the newest man You guy I believe He'd go Chris getting some Chelsea here man And on the back we have the 250 meaningful material Of Nuno Mendez PSG Alrighty guys that will do it For a break here We'll do a quick auto recap Then we'll get going on to EPL choice next Appreciate you guys We had to 50 Mason Mount Jersey Auto 75 Modric Modric to 299 Joe Felix to 99 Reina 250 Hungman Sohn on the Jersey Auto We have Fiora Hashi Auto Jack Graylish to 99 of Georgina Got Gerson Sandra Tonali Got Defoe Got 225 Diego Jota Nice one there Liverpool Darwin Nunez Johansson We've got Thomas Muller Jersey out of the 50 To 75 Kai Havertz We've got the 50 Ansu Fati We've got DVD to 199 Paul Vonner To 25 Adiyemi Got Trent Alexander to 199 We've got Harvey Elliott Gonzalo Ramos Vardio And our gold frame auto to 99 Was Frank Lampard Nice one there Chris Alrighty friends that will do it For Champions League Museum Collection We'll get some EPL choice going next


People talk often about their family bloodline. I understand the enemy will strategize against you according to your family bloodline. But don't forget this, the blood of Jesus is more powerful than your family bloodline.