Thank you very much I'm delighted to be here at the at the Naval War College and This is a very difficult subject to Wrap up in a short period of time, but I'm going to do my best I have my wife and timekeeper over there. I'm going to talk for a half an hour and When I go over that I always go over She's going to signal me and then I'm going to spend the rest of time answering your questions and receiving applause and stuff No, nobody nobody disagrees a thing Anyway, the title of this book is revolution on the Hudson, New York City and the Hudson River Valley in the American War of Independence But the book itself is about the entire war because In my estimation the Hudson influenced the entire war the war hinged on on the Hudson just as general Washington thought it would just as King George thought it would one thing that the British and the Americans agreed on was that the Hudson was the key to The war and I I'm going to start by by pointing to the two key battles of the war one Saratoga Which there were really two battles of at Saratoga the last one was October 7 1777 and The second battle is the battle of Yorktown, which are also familiar with and that was October 19 1781 Now it's it's obvious that the battle of Saratoga Is part it has to be part of the story if it's about if it's about the Hudson and the American Revolution It's not so obvious that Yorktown is also linked To the Hudson very substantially, so I'm going to try to make out the case that The British obsession with the with the Hudson River Valley Played a key role in their losing Yorktown So my book it won't be released until June 14 I'm waiting for my critics because this is a new way of looking at the war and there's a lot of guys out there Who who study and write about the war so we'll see what they have to say about this Landish point that I I am making Okay, so let me go back and explain why The Hudson was so important New York City and Hudson was so important. It's because it was the start of a Land sea corridor from from New York to Canada and when the British war planners In London we're looking at the map. They said they thought well, we just cut off the colonies The northern colonies in New England from the south and we've won the war because the the most radical part of The revolution the radicals are concentrated in New England. So if we can create a Sort of barrier From New York City up to Quebec Then we will have we will have won the war and this is going to be easy to do because we already control Canada We already have the the Royal Navy and so the plan was the the the war plan Was to to Have two armies One one coming from New York in the south Up to Albany and one coming down from Canada to meet at Albany so they these two armies were going to meet at Albany and This miraculously was going to cut off New England From from the colonies to the south which were not so rapidly Anti-British as as the New Englanders were and this would end the end the revolution Simultaneously with the with the meeting of these two armies the British Navy would have imposed a tight blockade along the Along the New England coast and really seal off New England. This was there. This was their vision Now interestingly enough All of the policy makers in Britain and all of the policy makers in the United States thought this was true That the British could do this and it would indeed Win the war Not only that Every historian writing about the war and they began writing about it almost almost when it was When it was finished Till this present day have thought that yeah if the British were able to do that they would have won the war One of the people who was most most influential influential in agreeing with this was our own Admiral Mahan who in his In his thinking simply agreed with with everybody else so In my book, I'm saying that this was never possible was never possible for the British to to do this for lots of reasons but Primarily because there was no political dimension to their strategy And New York was divided politically the the Counties around New York were pro-British not a hundred percent, but Majority of the people Were pro-British stat Staten Island was probably 90 percent British Westchester County was probably half half So Once you went north of Westchester County these counties were very similar in their political thinking to to New England and so the strength of of the Defenders let's say was going to come from these anti-British colonies just as they had at Lexington and Concord and in large numbers So if we come to the Battle of Saratoga for instance What defeated Burgoyne the British the British general Was the fact that the whole countryside was up in arms against him and that hundreds then thousands of militiamen were Were turning out to make become join the American army The American army at Saratoga When the decisive battle was fought was 15,000 Burgoyne's army had dwindled to around six thousand these these numbers of More or less, okay, we never have any exact figures on on these any Anyway, don't that's a separate subject of mine, but anyway be skeptical of all historical numbers But anyway, this is roughly what it was and Burgoyne wrote back to to London He said I don't have any support here. We were looking for some for loyalists to join my army They never appeared, but the Americans appeared appear by the thousands. He said this is Burgoyne This is the British general. This is why he thought he got defeated and this is why he He did get defeated so What was missing from British strategy was was it was there was no political element to it They thought they could they could Take over this area and defeat the Americans with military force alone It was too big a territory There were too many of us And in addition they wanted to do it on the cheap. They never wanted to commit the forces necessary To accomplish What what I don't think they would have ever accomplished they could never have defeated us with just an army and Navy alone with no With no political dimension Okay, I Want to fast forward to Yorktown and I'll fill in between if you want but Yorktown is October 19 1781. How is This book a flawed British strategy connected with Yorktown to me Yorktown had two key elements one was a naval battle September 5 1781 off the Chesapeake Caves the other was a land a land battle and That was the final one October October 19 now the British lost the The naval battle to the French on September 5 and I Believe the reason they lost the battle is because they didn't have their best Best admiral who was George Romney in charge of the fleet? Where was Rodney? Rodney of course was in Bath What was he doing in Bath having a good time? What else do they do? I mean not Bath, Maine, but Bath You wonder if you understand Rodney Rodney the previous year Had been in in New York And I will come back to if you want why I thought Rodney would have won the battle off the capes of Chesapeake capes against the against the French, but Why wasn't he there? He should have been there because he was At that moment he was head of the British fleet in America and the West Indies both was a combined It was a combined command In 1780 he had been in New York in September of 1780 He he just appeared in New York with ten with ten Ships of the line They weren't expecting him, but it gave the British then unexpectedly a Overwhelming force right in New York right then and guess what was happening then General Arnold was turning over West Point right then So Rodney arrived just as Arnold was about to About to turn over West Point and General Clinton who was the head of the British Army in North America had courted in in New York Now good old General Clinton was a Difficult man to to get along with I'm not going to call him a no difficult He had to work with another Admiral His at his naval counterpart called Abarth not and if Clinton was difficult to get along with you never met Abarth not The two of them hated each other terribly So here here Clinton is Wondering what the hell is going to happen When Arnold turns over West Point and he cannot get his troops from New York up the river Because Abarth not isn't cooperating. How do you like that? And that was a that was what was going to happen Until Rodney showed up and when Rodney showed up it saved Clinton. It was going to make the whole turnover of West Point Possible now for for Clinton Rodney had no idea any of this was was in the works And when he landed He was overjoyed to find out all of this stuff that was happening because he was now going to win the whole war of independence He he was going to win the war He he thought that he thought That the old British strategy of cutting off New England was absolutely the right thing to do Clinton Thought so too and they thought if they captured West Point They would then control all of the Hudson and somehow or other miraculously they would control The whole Lansing corridor all the way up to Canada. Okay Rodney bought that completely All right, and he's all set to go I mean he's he's all pumped up and they have the Troops and everything and Arnold's ready, and you know how capable Arnold was. I know I know was very capable so Arnold you you know that Arnold got captured or not being Arnold got discovered, but Washington couldn't catch Arnold. I don't go back to New York. So This is in September now of 1780 when Arnold gets back to New York What's the first thing he wants to do this a guy is a great political operator, Arnold He wants to make friends with Rodney And he wants to make friends with with Clinton which he does very quickly Rodney falls in love with Rodney loves Arnold So Rodney says okay. Well, let's go. Nothing has changed Let's go and attack West Point and we will win the war Seemed seem reasonable and Clinton himself Had been advocating this for a very long time and then all of a sudden Clinton Says no, I'm not gonna. I'm not gonna go ahead This the whole business is is ruined Washington's going to be Be able to have enough strength there to foil us. So Clinton stops this whole operation that Rodney thinks would have would have won the war because he he was totally It's a believe a believer in the old strategy of cutting off New England, okay Let's go back to Yorktown now. Okay, because Rodney goes back to West to the to the West Indies Very very mad and angry writes a lot of letters to to to London Saying you got to get rid of Clinton. You got to get rid of Abbot than off get some better leaders there Okay, so now Yorktown is only the next year and Rodney Is thinking that that the British fleet in the West Indies has got to come up to the Atlantic coast again just like he did the previous year and and They believe that the way things have Configured that there's going to be a showdown finally with the French in 1781 it's either going to be in the Chesapeake or In New York and Rodney should be leading the British Fleet, okay Rodney decides he doesn't want any part of this Okay, he's not coming. He's not coming. So he creates a what I think is a phony excuse and And is allowed by the Admiralty to go home and he leaves command Which results in leaving command of the of the British fleet in America to a Guy who under the circumstances really cannot defeat the French He just is doesn't have the ability to and he doesn't and the British lose The battle off the Virginia capes which result in the loss of of Yorktown Rodney himself when he reads about what this what his replacement Did at the battle in September 5 is Furious, I wouldn't have done this. I would have done it differently and so on and so on and he would have his his he in my estimation would have Defeated the the French Admiral and the whole war would have would have gone differently, so one they didn't get mixed up with all of this He went home. Why because he had bought that the whole mythology of the of the Hudson River Valley that the British had That if they took West Point, they'd win the war. They control the Valley. They control the corridor to Canada Which in my estimation is an illusion. So this is the connection I make I make between Saratoga and Yorktown So there So I I how we doing on the time fine, okay What time is it actually? 1220, okay, let me let me just a little bit more on on the wall. I in this book. I start really in 1776 I Also go back and and have something to say about 7775 to so the book and I I have something to say about what happened after Yorktown so the book is a book about the whole a whole war, but the British The British learned the lesson in 1775 and 1775 they got their tail whipped as you know in a number of battles Lexington and Concord was was just the beginning the reason they did is they didn't commit enough troops they always thought they were going to Defeat the the American Patriots on the cheap To begin with Lexington and Concord they had less than 4,000 troops in in Boston and half of them had to be used to keep the people in Boston down So they had they had like 2,000 troops. They were sending into a countryside that that turned out 20,000 militiamen, okay So they they the king learned his lesson he thought in 1776 and he committed 32,000 to the invasion of New York, which began in August of 1776 and the Canadian army that was coming down from Canada was 10,000 there's 42,000 General Amherst who had who would won the French and Indian War for the British here thought you needed at least 75,000 I Don't think 75,000 was enough Benjamin Franklin thought well more than that But anyway, it turned out that the getting together the 42,000 10 from the north 32,000 from the south it Just wasn't going to do it and and the the two the two commanders British commanders in New York recognized this Admiral Admiral how the the Navy chief and General how who was the the army chief they originally thought that yeah This number will be enough. They weren't enough the fleet wasn't big enough either so But as they got into it they could see that that this was this was they were was woefully inadequate so at the end of 17 1776 General how sent sent home a new plan a new strategy for 1777 1777 He thought that he now knew how to win the war I mean, you know the in those days you didn't fight in in the winter time and besides which he'd like to have a good time in New York which he did and and so he he came back out in to do battle in in the spring of 1777 but the king was having no pot of Of of increasing the army anywhere near the extent that how believed he needed and I think how was how was right and the The first sea lord the guy who ran the ran the Navy for the British He he thought that the fleet was just fine the way it was And they they they didn't need anymore and so The whole operation in 1777 which was a replay of 1776 for the British failed guess who the king blamed Not the strategy he blamed the general and the admiral and He blamed the people carrying it out not not his his vision so So Far as the strategy this strategy is concerned it's sort of it sort of put On the side because the French came into the war because of Sarah the victory at Savitoa in 1777 the French come into the war and and so it becomes a world war and so the strategy Changes nonetheless There's all these people including Clinton and then later Rodney who still are saying if we only Carried out the old the old strategy. He never disappeared. It wasn't wasn't it wasn't still had a Lived on they thought if it only we had gotten to Albany can you imagine getting to Albany? What anyway? So once the French come in it makes it possible for for the United States to actually defeat the British which because of their own internal feuding was accomplished that at Yorktown if you knew if you knew the the American army and what What it had deteriorated into by 1781 You we would wonder how could we possibly Carry on without the French where we couldn't And the British the British simply if they had stayed here and waited us out would have we would have Would have Well, I can't say won the war because the war would have stopped And there would have been a negotiation between the French and the British and Lord knows how that would have would have turned out but if If the British divided the territory somehow rather with the French Whatever Britain controlled sooner or later later on they were going to make the same mistakes that had brought about Lexington and Conkin And the whole thing was going to start up again Finally George Washington I've studied George Washington for a very long time and He in my estimation was one of the greatest Leaders in world history if you asked me to name the six most important figures in world history He'd be one of them if you if you examine what he did up close during the revolution all this time Not to mention what he did afterwards, but just just that holding all of this together I mean he was holding it together He was amazing and He was a great writer. He'd written so many letters That he had developed a great facility With the language and hit all of his correspondence During the war boils down to ten ten volumes We've had three versions of them. I have the first one that was published in the 1840s And I've read them on astonished at what a great great writer he He was and his wife I want to kill If this if there's anyone I want to Dig up and shoot again. It's her because Because naturally he wrote wonderful letters to her and they were a wonderful couple You know and she's joined him every every year in the winter time and in the most awful Circumstances in the battlefield She then goes and burns all of the letters Every blessed one of them except for two and the two of them are fantastic. So She I'm gonna get I Don't know how All right, so I'll stop there and answer answer and then questions you have See no questions that answered all the questions. Do you address the various battles around the New York area? Yeah, many and they lost them all. I mean I do I go through I go through every battle yes, but Doesn't have nothing original That I have to say about those battles. I hope I'm accurate, but there are guys who have studied them in great detail And if you want to know specific battles My my summary might be a refresher course for you I Have spent Really over 20 years studying this Business, I'm a grind and so I think I have a very good summary, but That's what it is. Okay, like the Battle of Brooklyn Which is one of the great battles of our of our history which we Which we lost I described that but there are the many other people who describe it in in greater detail One of the things that I emphasize in the book is Prisoners how the British treated American prisoners and I have a I have a nice section in In the Battle of Brooklyn about what they did with those prisoners, but there are other books Not too many about the prisoners. We're so close to the British That we don't like to think about what they did to our prisoners and what they did was a war crime There were 30,000 American prisoners and they killed half of them And there's mean there's no blinking it, you know, this is this tap in Washington was after them to stop it right from the beginning Letter after letter it didn't it didn't have any effect John Adams. You name it so I have I have attached that to every every battle What happened to the prisoners? Yeah, if you say that we're going down there was both the tactical and strategic mistake both are losing and For strategic they bring the friendship What would you have done with those 8,000 troops? I do If you don't think the Hudson was a value, what would you have done those 8,000 troops? I would have brought him home I would have I would have I would have brought all the troops home In 1775 They had 4,000 troops in Boston, I would have brought those 4,000 troops home One of the great English states and William Pitt who who ran Who ran the French and Indian War in England and won the French and Indian War when When the time came crunch time came in 1775 when the government was making up its mind About whether or not they were not going to negotiate with the Americans or they were going to go for for force of military military option He said he said to the king take the troops out of Boston and talk with them The the issues can be resolved If you go to fighting It's going to be very iffy. There were a lot of them They know how to fight they fought well during the French and Indian War despite what your What your British officers? Will tell you they're not afraid of your troops and there's just you know There's tens of thousands. I mean how many Americans would turn out so that was his advice and it was good advice And and we never would have had that and then we would have had a British America happily the the people Who fought the British at Lexington Concord? Okay, those those guys and women the women were a big part of that story They had the highest standard of living in the world Okay, of any ordinary people in in the whole world They did not want to fight. They were living quite well It would it would have been very easy to negotiate with their Continental Congress, which they With which they produced this was a war that never should have been fought. There are wars like that as You know, we won't get into that but but So that's not what you wanted to hear I Here's what here's what here's what Burgoyne thought Burgoyne thought that he would get to Albany quickly He thought that he That all these loyalists these this mythical Army of loyalists would come to join him He might get together 15 even 20,000 troops himself, you know without reference to the army coming up from New York He'd base himself in Albany He'd have plenty to eat there and he would take that army and march into Massachusetts and Connecticut. There would be loyalists that would join him and he would win the war on his own and He thought in 1777 That by Christmastime anyway that he would be the victor and the great they you know He would be the the Nelson and the Wellington and so on the great hero of That's that's that's what he he thought I think that was a pipe dream Can you imagine if he took these people into? Massachusetts and you had the farmers coming out again and from Connecticut and then the farmers from New York and all the all the All of them north of Westchester County and how big though that army would be he'd be fighting for a while there Yeah You know on the first question, so it sounds like when you if I followed what you said you're suggesting that Although this a central premise of your book is that The misguided strategy of control of the Hudson would lead to winning the war Was it was a fallacy? I think you're applying perhaps That there wasn't a better alternative strategy that would have worked But but in fact the waging of war in the first place was the wrong tool to achieve the political objectives They wanted that about absolutely. That's a William Pitt thought and Who could disagree with William Pitt? Yeah, it's not it's it's it's it's hard to think of that, you know in all the waste on both sides and It's like brothers fighting it was some people called a cousin's war and so on So it's hard to think that it was on just unnecessary.  Exactly. Exactly. That's a very good point They the British found out All over in the south same thing, you know, what do you mean we got a fight for you? What do we have you for Question was I was wondering if you had looked forward to the war of 1812 and the same strategy that the British looked at during the war of 1812 well, I I Wrote a book on the war of 1812. You've got to get that book. It's just Well, that was that was four years ago and and even even John Hattendorf stayed awake for that Sorry, you never know I hate to get off in the war of 1812, but but the the the British it seemed, you know emotionally Were mad at us. We they never got over the war of independence and The the war of 1812 almost seemed inevitable at some at some point But their whole their whole inability to respect us militarily Which was very evident in 1774 and 1775 strangely continued on Up to the up to 1812. I mean all the things that they did to us Leading up to the war to make us angry enough to actually the clear war on On them was was an outgrowth of the fact that they didn't they didn't respect us militarily well We had been debating ourselves whether or not to even have a Navy For all of this time our Navy and in the in the Revolutionary War had done very poorly There was no Navy after the Revolutionary War until 1794 And then there was a great debate in this country about whether or not we ought to even have a Navy and the the irritation with the British was primarily on the on the water, so They didn't respect us because we couldn't make up our mind did we want a strong Navy did we want a strong army or not, you know because a lot of our political thinkers thought that Having a strong military would be dangerous It would it would you know some man on horseback would become a become a dictator So the war of 1812 surprised them that we did so well particularly on the water even with the minuscule Navy we had and the British leadership of that time The Prime Minister was Liverpool and the Foreign Minister was castle raid They were they felt Towards the Americans as everybody else did they had no respect for them militarily, but during the war During the war and I can mention a number of places where the Americans unexpectedly won, but New Orleans is a good example Castle ray Said That you know If we don't make a real peace with these people We're going to be fighting them for the next hundred years. I just think of it the whole border with Canada We're going to be this you know, you can how about the southern Border how about the sea and so on all these all these things so he he got Liverpool Liverpool wasn't was He should he should have been running today He was a kind of politician who knew exactly what button to push to get public support. So He got he got Liverpool to Support a total revolution in British policy towards the United States at the end of the war of 1812 and from there then on They treated us with respect and guess what James Madison who was the president and Who had been one of the people back in 1794 who did not want a Navy and wanted a very small Army He had changed his mind too. He thought if we don't have respectable force Look, what's going to happen to us? Okay. He thought we need a respectable force in order to protect the Constitution It wouldn't be a it wouldn't be a threat to it. So the American military for the first in time Well, anyway, the American military was not disbanded at the end of the war And certainly not the Navy We he who kept the Navy and that's really a lot of guys in the Navy Admiral Admiral granite who loved my book Thought that the War of 1812 was enormously important in the history of the Navy because of Our country finally agreed. We need to strong Navy That's that's pretty long-winded answer to a very simple question right now. I'm sorry Can you imagine being married to me? I mean she's already Yeah, so you're saying animal Rodney was committed to the strategy Yeah, and he didn't know anything about it He didn't know anything about the United States, but he he bought that whole thing and you know who was encouraging him Arnold Arnold when I'll get to New York. He made a point of getting close to Rodney and Ronnie Rodney fell in love with him literally and Arnold convinced him that this whole strategy Would would work So he was committed to the strategy, but he still didn't want to have anything to do with engaging the French fleet well, no, he wanted to engage the French fleet, but he didn't want to do it personally because He'd be working with Clinton Clinton was the guy in 1780 Who who stopped him from taking West Point Rodney thought that he and Arnold would easily take They had the troops there they had them on the transports, you know The the Americans were discombobulated by what happened with with Arnold. I know was a great general He was a He was a guy who I mean he was a strange character in love. I was I would say in love with money More than anything else, but he was he was a very very good military Leader and Rodney had a lot of respect for him Rodney himself was one of the greatest fighters ever So yeah, they would have probably taken West Point. I don't thought I would have taken about 10 days And then what would happen? You know Beyond that what happens well gets a little fuzzy, but West Point they thought that was the they really thought that once they took West Point Washington would just give up that was that was the hope. Yes, sir Were there some historical cases that they were thinking about them that made this so compelling an idea They were they looking back in history whether it was back to classical Period. Oh, no, no, no, no law knows what they were looking at I I can't answer that question it it the distance between New York City and Albany and then distance between Albany and Montreal and So on and so on and the size of the the Royal Navy And one of the great greatest naval leaders was Admiral Lord Howe Who was the fleet commander? Here at that time he he he didn't say so publicly, but he He thought he thought that whole strategy was foolish, you know, he never he never wrote about it So this is just a supposition of mine, but he wasn't surprised at any of this Had happened They let him go. It was a really odd thing. They're really the best admiral Nelson called how the best Admiral they ever had Nelson and and the the the first the first sea law the head of the The Navy sandwich didn't like how Okay, he didn't like him how for a lot of reasons But he made life difficult for how And how how went went back home And said my sandwich was very happy to have him Go back home in in the fall of 1778. Okay, if how had been kept on He probably would have won. He'd be he he would have been better Equipped than Rodney even to win the battle off of Chesapeake Chesapeake tapes the the the war leaders in London Were so smart They were happy to have how come back So but how was never in love with that strategy? No But if you asked me to give you evidence of this then I would Be too busy for you or something. I can't yeah, yes, ma'am What Lord how had when he first came in 1776 He had about 70 warships total, but they were common and going all the time You know, they're breaking down. You had to fix them and so on we had limited facilities here Halifax was Was very very limited in what it could do some of them often had to go back to back to to England 72 was not not near enough. I listed my book the things that the Admiralty tasked how to do with these 72 ships. It was ridiculous and At no time in the war did the British have more than 90 warships here and it just wasn't enough You know, it seemed to be enough You know, that's a lot of a lot of ships, but it just wasn't this is a big country It was two and a half million people 500,000 of those were slaves, but still that's two million people and the slaves as the war went on became part of our omnis Except for Places like South Carolina or South Carolina would never have slaves become part of the army, but the other the other colonies Would so the the the sheer size of the military to do what they wanted to do over here They just didn't want to come to to grips with that it's like Lyndon Johnson trying to think how many troops were actually needed What was actually needed to fight the war in Vietnam? I'm the Vietnam War era. Okay, don't get me going on that one That's another story, but it was it was similar that it shouldn't draw parallels like these historical parallels, but Johnson never ever wanted to face the Political consequences of having having the size of the military that he would have needed to do what he wanted to do Well, you guys must study the video study the Vietnam War here. Yeah, okay. Good. Yeah Focus on so much time Well my wife and I were in business for a long time she's an artist and I I was a college professor up until the age of up until the age of 40 when I was totally burned out and I went into her business and After 20 years of that I I decided I wanted to spend my time writing again when I was a professor. I'd published three books and I wanted to go back to it I got over being being burned out and she wanted to she got tired of the business and wanted to to Spend full time with her painting, but but I I knew that it was going to be a job It was nice to have some money in the bank, you know when you're a professor you're broke I mean what they pay you was mind-boggling Anyway, so we had the money and and so we that's what I did I studied I studied for years and years and years And then my first book came out in 2008 and I this is my fourth book My fifth book will be out at the end of next year will be on Lexington and Concord when I was a professor. I wrote three books One That my first book was on the Cold War and we sold a million copies a Million copies is a lot you know this this day and age for for New York publishers who are only interested in money I Shouldn't be so hash on them, but The benchmark for for a history book like I write is like ten thousand copies ten thousand They cover all expenses, and they'll cover your advance and so on a Million is you know David McCullough has sells a million no anyway So the publisher thought whoa, I'm gonna you know So the next book didn't do so well 75,000 we sold which is very very good. He still made plenty of money So he's going hot in the next book Was on World War one and he he printed this humongous number in anticipation of big sales He couldn't we couldn't give him away. I Mean literally literally couldn't give him away. No one was interested in those days in world when I know But anyway after that defeat I decided I didn't want to go that way had it come back Sorry But the the when I get into studying early American history the thing I noticed Right away was where's the Navy? Navy's not in it When they when historians think about those ships and all the things they have to learn about them They get stomach acid they they think they have to They think they have to learn a new language which they do and the Navy is not in these histories I mean, it's amazing and a little bit that is is is just you know, we would make you cry And so our kids grammar school all the way through Learning their history without the Navy in it So you wonder why people don't know about the Navy and don't appreciate all the things the Navy has done So I began to focus on the on the Navy and integrating the old story that we tell with the with the naval story but it's a It's an uphill battle Because the amount of ignorance about the Navy and of course It would make you cry how little is known about what how important the Navy's been since since to the world since World War two you know so anything we can do to to in the area of public awareness is is worth doing because So True and it's very dangerous that the people don't know how important the Navy is very dangerous because different things like budgets Now these people cutting budgets and so on You can cut the wrong place What do we got here all right five minutes anyone want to want to refute what I'm saying Not about the Navy and public relations that drives me crazy that subject Yeah, what was the influence of the common of Congress? Very similar to the Congress today it would drive you crazy The politicians politicians are politicians. They don't They don't change they they represent the areas where they come from And so far as the immoral character is concerned don't expect a high level of In that category they represent who they represent and they may think that the constituents Mistaken but you never know that you might hear it privately. There was a congressman who was a a friend of mine from Iowa named John Culver lady became a senator And John Culver was was in the House of Representatives He was a big proponent of foreign aid foreign aid was was Was always controversial, but One guy from I don't know. I will know what state I think it was Alabama One guy gave him a particularly hard time always in the house when the bill foreign aid bill came up This guy was 300 percent against foreign aid, but foreign aid would all would always pass nonetheless But foreign aid got progressively less popular. So each time it was voted the vote got closer So on this particular year, I forget the year The foreign aid bill came up and they were they were discussing it and the vote was about to Happen and this guy Mr. 300 percent against the foreign aid comes rushing up the Culver and says to him John it might not pass And and you know Culver who was about as cynical as the rest of us couldn't believe this one and He he loved to tell that that story. So anyway Congress back then is a mystery and it's a mystery too It's about as I understand about as much of it as I do current Congress The way we treated our military in those days, I have a little bit of it in the book You would not believe Okay, you just you just wouldn't believe how badly we treated and particularly our enlisted men And and this was all simply political You know, these were the kids who fought the war all the way through sacrificed everything And the way they treated Treat them well, they didn't pay them and all these these things. It's a it's a long awful story, you know, but So no, I don't think the I don't have any great admiration for the for the Congress and I place a huge Ephesus on on Washington And they always had Washington, you know who who would do the right thing But don't ask them to do the right thing Well, look at well Yes, anything else


A Muslim sees the unseen. A Muslim sees the unseen. And if we don't see the unseen, then we might fall victim to some ideas that will attack our Aqida. But a Muslim sees the unseen. That's how Allah described us in the Quran. So we interacted the unseen. And part of that means that what we're doing right now, we're seeing the invisible reality of what we just did. What we just did, there's a beautiful verse in the Quran that God, Subhanahu wa Ta'ala, he tells each of us. He's telling each of us. He says, no person knows the delights of the eyes hidden for him or her because of what they used to do.  Hidden for him because of what they used to do. So we need to keep our eye on the prize. Ramadan's ending, not yet. We still have a beautiful night, another day or two. Don't check out yet. So you have some Quran to read, some du'a'at to make. There are some people that you haven't made du'a'at for yet. Remember them by name. But Ramadan insha'Allah is going to end soon. Ramadan is going to end soon. But we will continue insha'Allah. And the Rabb of Ramadan will continue after Ramadan. The Rabb of Ramadan is the Rabb of Shawwal. The Lord of Ramadan is the Lord of Shawwal. And so when we finish this month, we keep our eye on the prize. We keep our eye on the prize. When our eye slips off the prize, that's when the dunya gets heavy. The prize is Allah and the next life. And when the eyes are on the prize, the eyes of the heart and our eyes are on the prize, then we can ride the waves of the dunya. But when our eye slips and we start to see all these other things, then the dunya gets heavy. And then we need to remind ourselves, hold on, hold on, hold on, hold on. Let me pause the world to be with the creator of the world. Let me pause the world for a moment. Let me get, let me check back in. Let me check back in. Alright, eye on the prize, Allah. You know, yes, this person just bought a fancy car that I can't afford. Yes, this person just got a promotion and I didn't. Yes, this person is married and I'm not. Yes, this person is pretty and handsome. Handsome, prettier than me. Yes, hold on, hold on, hold on. Let me get back. Allah. Keep my eye on the prize. Ya Allah, I live for you. Ya Allah, I breathe for you.   I trust you. Keep my eye on the prize and ya Allah, I'm with you and I'm going to be patient for a few days here. And then you made a deal with the Prophet, Sallallahu Alaihi Wasallam. Each of you and each of us have a deal with Allah and a deal with the Prophet, Sallallahu Alaihi Wasallam. And I'm going to tell you these and then we're going to go do the special prayer, Sallallahu Alaihi Wasallam. Each of us have a contract with Allah and a contract with the Prophet, Sallallahu Alaihi Wasallam. We have a contract with Allah when he made you and created you and blew from his, in a manner befitting to him from his spirit, from his Ruh. Right?  And you said, Bala, yes you are. So you made a contract with him and that contract is to obey him. And we either obey and Islam is a debt. Religion is a debt to God. Deen is a dain. Religion is a debt. We owe God prayer. It's not a, we owe God. We either do it in this life or the next life. But it's his. We owe him five prayers a day. We owe him five prayers a day. We owe him zakat. We owe him fasting Ramadan. We owe him. We either do it in this life or the next. Right? So we have a contract to obey Allah. And then we have a deal with our Prophet, Sallallahu Alaihi Wasallam. Right? When they went to the Aqaba pledge and they pledged allegiance to the Prophet, Sallallahu Alaihi Wasallam. And they put his hand in his hand. Right? Sallallahu Alaihi Wasallam. And they said, and he said to them, will you pray? Yes. Will you be honest? Yes. Will you true? Will you protect me? Yes.  Did you hear that guys?  He said, yes. He said, okay, if you do that, then what do you get? And what do you get? You get one thing. What do you get? You get paradise. That's the deal, ladies and gentlemen. You made a deal with the Prophet, Sallallahu Alaihi Wasallam. And you know, when we collect that deal, we collect that deal at the fountain, at the day of judgment, and he's waiting for us, Sallallahu Alaihi Wasallam. He's waiting for us. So we fulfill our part of the deal. We get paradise. That's the promise, ladies and gentlemen. And Mohammed Constantino, I love you. Catch it, bro. Catch the kiss. There you go. Put it on your cheek, down on your cheek, bro. There you go. I love you. So that's our deal with our Prophet, Sallallahu Alaihi Wasallam and our contract with Allah. May Allah be gentle with us. May Allah bless all of you and all of us and accept from us. We're going to do a very special, unique prayer with Amm-ul-Tarif. This is in both that I know of, that I studied in the Hanafi and Shafi'i Medheb schools of thought. It's called the Tasabih prayer. It's a really special prayer. Amm-ul-Tarif will describe how to do it in the main hall. And it's a very, very nice prayer. If you've never done it once in your life, do it at least once in your life, because we want to do everything in our books and everything in our tradition at least once. So we go to the Prophet, Sallallahu Alaihi Wasallam, and he's proud. It's the Tasabih prayer where within the prayer, Amm-ul-Tarif will describe. We do Subhanallah Alhamdulillah, La ilaha illa Allah Allahu Akbar. 75 times each raka'ah for a total of 300 times. May Allah accept, insha'Allah.


The sun is continuously delivering energy to the earth. Interestingly, the amount of sun energy is larger than the amount of energy needed by all people using current electricity on earth. How can we efficiently use the energy of the sun? One way is to use solar panels, which will result in direct energy, which are a large amount of energy on sunny days, less on cloudy days, and basically nothing during the night. Of course, the human energy need is not directly reflected in this variation of sun energy during sunny and cloudy days. So we need a way to store the energy. Trees are good in storing energy. During photosynthesis, the pigments in the leaf convert the sun energy into chemical energy by converting carbon dioxide and water into sugar. Sugar is stored in the trees and the sugar is a form of chemical energy. When we burn the trees again, the sugar is consumed again and energy will be released, which we can see and feel when we, for example, enjoy a campfire. Unfortunately, the process of photosynthesis is very slow, so it will take a very long time before a tree is useful to give enough energy to be useful. An alternative way to store the sun energy is by producing hydrogen. In this type, the sun is used to split water into hydrogen. Hydrogen can be stored and subsequently the hydrogen can be burned and the energy can be released, which can, for example, be used to drive a car around. Moreover, oxygen is produced as a side product by generating hydrogen. Moreover, if the hydrogen is burned again in the car, for example, oxygen is consumed and water is released. So it is a very clean way of energy. The only way that comes out is water. So how is now hydrogen produced? So water can split the sunlight into oxygen and hydrogen, which has been discovered by Fuyutzima and Honda roughly 40 years ago. However, this process doesn't occur by itself. A catalyst is needed. At the moment, this process is still very low in efficiency and therefore not used on large scale. For example, it is not known why one catalyst works better than the other. Many scientists are trying to understand this process, but they are doing this basically by trial and error process. What we do is a different approach. We try to really understand the mechanism. So we can really look at the process during the processes happening. How is water split into hydrogen and oxygen? So how can we do that? So with my ERC-Finnert research group, we use newly developed laser-based spectroscopy methods where we can only look at the water molecules present at the interface. We don't see the much larger amount of water molecules present below the catalyst. For example, we can see how the water molecules bind to the interface. And we can try to link that to the efficiency of the catalyst. The Sun needed to initiate the water splitting process is in our case mimicked by a laser pulse. But what's subsequently happening? To understand that, you want to look at the process at the time scale the process is taking place. This principle has nicely be illustrated by Moebridge in 1878. At that time, there was a big debate going on if a horse is during caloping once with always four feet from the ground or that it always has one foot on the ground. So Moebridge could show by fast camera that a horse is basically flying during caloping. So we use the same approach. However, as molecular processes are taking place on picosecond time scale, we cannot simply use a fast photo camera. So we have to use laser-based methods with sub picosecond laser pulses. Where we use one laser pulse to initiate the process and the second laser pulse to make pictures of the moving molecules. So in this way, we can obtain a movie about how this photocatalytic mechanism is working at the interface. If we understand how the process is working, hopefully new and better-designed catalyst can produce so that the hydrogen production can be more efficiently, so that sunlight can be stored more efficiently. Hydrogen produced in this way is a very clean source of energy and environmentally friendly, which can be used, for example, for public transport, for driving a car around or for electricity and heating at home. If hydrogen can be produced more efficiently, we don't need the large amount of fossil fuels as nowadays, but we result in less pollutants and therefore make life a better place to live on.


Okay. All right. Oh, it's an echo. Got it? Echo, echo. Okay. Uh, test, test, test. Did it go away? Good. Okay. Yeah. Okay. Great. All right. My name is Joshua Watt, and I'm going to talk to you today about the software, bill of materials, and the supply chain with the Octa project. A little bit about myself. I've been working at Garmin as an embedded software engineer since 2009. And we've been using Open Embedded in the Octa project since 2016 to make embedded Linux products. I'm a member of the Open Embedded Technical Steering Committee, and there's all the various ways you can contact me if you would like to do that. All right. So if you are unfamiliar with the Octa project and Open Embedded, Open Embedded is a community-driven project that provides the Open Embedded core layer and BitBake, which is the build system used to be build primarily embedded systems, but all sorts of other things as I'll talk about. The Octa project is a Linux foundation project that maintains the Pocky reference distribution and also provides the auto-builder hardware used to run a whole bunch of QA tests to ensure that the project has high quality. They also manage a release schedule, provide funding for personnel, and provide a lot of excellent documentation for the project, which is awesome. I really like our documentation. You should go check it out. So here's a brief outline. I have a lot of slides, so I'm not going to stop here. All right. So what is the software supply chain and why do I need one? So the software supply chain is really about answering what's in the software that we're shipping. So we ship a binary to someone or use it ourselves. We really want to know what's in that thing that we're using. So we want to know basic questions, like where did the software come from? What version is it? If we have licenses, we need to know if we're complying with those licenses, be it GPL requirements or attribution requirements or things like that. We also want to know if that software has been tampered with, either maliciously or unintentionally, because we don't want to expose ourselves or our customers to unnecessary risks by having tampered software. And the same thing with exploits. Like if our software is vulnerable to exploits, we want to know that to protect ourselves and our customers. Ultimately, the software supply chain tries to answer the question, can the deliverables that we're shipping be traced back to the code that generated them? So at its core, that's sort of the intent of the software supply chain. So I'm going to talk about the open embedded build flow, which is how open embedded builds software and how the way that it builds software has an inherent software supply chain to it. So when people want to build stuff with open embedded, they primarily start out with three things. They have the source code that they want to build, which comes from Git repositories or tar balls or whatever it is. They have some metadata that we call recipes, which describes how that source code should be built. And then we have various policy information, which is like, do I run and run system D or sysvnit, you know, various configuration bits and bobs that make your thing do whatever it's supposed to do. They feed all this into this magical tool called BitBake, and it spits out what we like to call a target image, which can be a whole bunch of different things that I'll talk about in a minute. You take that target image, you put it on your widget, and you profit, right? It's great. And when we talk about a target image, there's actually a whole bunch of things that we classify as a quote-unquote target image that you might not normally think of as a target image. So we do have the traditional embedded flash this on an SD card and boot it on a Raspberry Pi, like you can see up there at the top. We can also generate microcontroller firmware that you could flash onto a microcontroller for whatever reasons. But we can also generate a whole bunch of other stuff. So we can actually generate images that you could put onto a hard drive and boot up a full PC like any other desktop Linux distro. And we can also generate virtual machine images, primarily QEMU, but I actually learned just this week that we can generate virtual machine images that you can import as AMIs into AWS and run them there if you want to do that. Another lesser-known thing that we can generate is OCI-compliant container images. So we can actually build container images that you can import into your favorite container runtime, be that Docker, Podman, Cryo, whatever the flavor is this week, and run those. Now, we can also generate package feeds for the various package runtimes that we support, which currently is IPK, Debian, and RPM. So these allow you to publish package repositories just like you would for any desktop distribution. So if you've, say, flashed your image to your Raspberry Pi, you can then point it at the package repository that you've published and just install software using app or DNF or whatever you would like at a desktop distribution. There's also a whole bunch of internal things that we can generate. The SDK allows you to... Is it something you can ship to customers or use internally for yourself that allows you to compile software or provides compilers and tools and things to compile software against a given image so you could compile a executable, copy it onto whatever your target is, and then run that. We also have the extensible SDK, which is a more advanced version of the SDK that I really don't have time to go into. And Build Tools Tarball, which is really cool for supply chain reasons, and I'll talk about that at the very end of my presentation. So digging into how the build flow works that BitBake provides, so we start with a couple of different things. So over there on the far left, we've got the host tools. So these are the bare minimum set of tools that you need to build the project. This is going to be things like Python because BitBake is written in Python. Git and host compiler. So this is going to be the host GCC needed to compile stuff to run on your host. It does not need to be a cross-compiler. We also have some source code up along the top there. And we've got the recipe metadata that says how we're going to build that source code. So the first thing that BitBake is going to do is it's going to take these host tools and it's going to ingest some source code and process some recipe metadata and produce what we call the native tools and also the cross-compiler. So we're actually building the cross-compiler that we are going to use later on as part of the build step and we're also building what we call these native tools. So a good example of a native tool might be the Google Protobuffer compiler. It's something that runs on your host that you use as part of a build to generate stuff that you need later on, actually on target. So, you know, you use the Google Protobuff compiler to compile protobuff files to C code or C++ code or whatever it is. Using these native tools and cross-compilers we're then going to process yet more recipe metadata and ingest more source code and this is going to produce the target packages. So these are the things that are targeted to run on whatever your final target architecture is, be it ARM or X86 or RISC-5 or whatever. The final step that we do is we have yet more recipe metadata that says what target packages you want to install on your final target image and so we've got a recipe that says install these packages on this image and produce the final thing. So the way that BitBake tracks when things need to be rebuilt is using a very sophisticated method of hashing. So the way that this works is all of the inputs to a given recipe are hashed together to produce a single final hash that we call the task hash and then this hash is then used as the input to subsequent recipes and gets incorporated into their task hash. So you get this chain of hashes all the way through the system and what this means is and that hash includes all of the inputs and all of the recipe metadata itself that is used during that particular build step and so if any of that changes that's going to change the hash and that signals to BitBake that that thing needs to be rebuilt and because that hash changes then all of the downstream hashes from that will also change and so BitBake knows it needs to rebuild all of the downstream stuff that depends on that. So just as an example, if the source code for the Google ProtoBuff compiler for example changes because you bump the version or something like that that's going to cause that hash of that source code to change. When that happens, all of these downstream hashes that are dependent on that are going to change so BitBake will then know that I need to rebuild the ProtoBuff compiler I need to rebuild all the recipes that use the ProtoBuff compiler and then I finally need to rebuild the target image at the end of the day. So because of this sophisticated method of hashing we actually have a really good traceability i.e. a really good software supply chain back to all the things that went into your image because we can take from a target image and trace back through all these hashes to the individual components that went into it and importantly, we can do this not only for the target packages the actual cross-compiled things but a significant portion of the native tools and cross-compilers because we're building them ourselves as part of the entire build process basically everything but the host tools we have very good supply chain tracking on all of these things just inherently due to the hashing mechanism that we use to track dependencies in general. Alright, so software bill of materials so the way that I like to describe software bill of materials is it's the nutrition information for your software if you've been to any talk here you've probably seen something like this already but basically, you know, we've seen these ingredient labels on food they're a standardized way that we can quickly see what we're putting into our bodies the basic ingredients that are in there and then various information about it and an S-bomb is kind of the same thing standardized encoding that allow us to easily exchange information about what's in our software and know what's going on with it and so there's multiple different S-bomb formats out there there's a SPDX and Cyclone DX another one that I always forget because I can't remember that might be used to describe the same actual software supply chain so you can kind of think of the S-bomb as sort of the way that you encode the information in whatever your software supply chain is at least that's the way I like to think about it so the important stuff that's in an S-bomb is what the software components that we have in our system and what are the relationships between them this is a really popular graphic that's used around to describe this and so it can show you the various properties of what's going on and how they're related and I really like this graph because I think it fits really well with this graph you can pretty easily imagine that you know, we have a recipe this is Carol's compression engine and then that is then ingested by another recipe as a dependency that's Bob's browser and we're tracking all those dependencies and all that information just like that so what stuff do we have in our recipes that we might want to include in an S-bomb well it turns out we actually have a lot of stuff so our recipes are fairly comprehensive in how they describe software so we have a lot of information we can include we've got the stuff that you would expect to be there so we've got the versions of the software that we're building we have the source code URLs from when we downloaded it which we have to have because we downloaded as part of the build process we also have pretty advanced license tracking that we do because for a long time the project has had tools to help people do license compliance, either GPL compliance or whatever it is just to know in general what licenses they have on the things they're producing so we've had that for a long time we also have all the build time dependencies which we obviously have to have in order to correctly build the software so we know what all the build dependencies are and similarly the run time dependencies we have mechanisms for automatically determining some run time dependencies or we have to manually annotate for other ones and we have to know what those are and they have to be correct or the software wouldn't run on the final target we also do a lot of CVE tracking so we track the CVE metrics for the recipes that we have and then if there are CVEs that come out we will patch them and we also have tools that can make it fairly easy to figure out if a given recipe is vulnerable to CVEs and things like that we obviously know all the source files again we downloaded the source code, we extracted it we did whatever with it so we have a pretty good idea of what those are and also the files that end up in the individual packages that producing we put them there and then we know what's in them and something that's a little different about all of this information is that we're very authoritative on this information like we know for sure these are what these things are because we generated them we're generating them from first principles which is a little different than perhaps you've seen from some other Sbom related tools where they do scanning of completed images and not to say that that's wrong it's a different way of doing it instead of we're not scanning things after the fact and trying to heuristically figure out what things are we're just saying this is what it is because basically we said so this is what we did and we actually have a currently our policy is that we won't include information in the SPDX if it didn't come from that first principles approach so we're not planning on adding heuristical scanning tools or anything like that because it's not really our place we're just saying what we did in our Sbom stuff that's a little different from what you might see in other Sboms generating Sboms is very easy we can generate Sboms in SPDX JSON format we chose SPDX as the ISO standard and also Linux Foundation project it just kind of made sense to do that and we do JSON because basically all of our build system is written in Python and JSON with Python is very easy so you initialize your build environment right here just like you would for any other build you add this single inherit plus equals create SPDX line to your local COMP file you bit bake your image it will generate SPDX for everything that's in that image and all of the native tools that helped produce that image so very easy basically what this ends up doing is it adds an extra step during the build process at various points where when it does a certain thing it will then generate an SPDX document that describes the information about what happened during that step of the build and then at the very end we take all of those SPDX documents and we put them in a big tar ball so that you have one file that you can use instead of having to deal with many files so what features do we have in the SPDX most of these are just a repeat of what's in our recipe metadata like I said we have pretty extensive metadata so it's pretty fairly straight forward to translate that to whatever SPDX field it happens to be there's a couple things that are worth highlighting so we do we populate the declared license field based on our license metadata that we have if a license is not one of the SPDX known license identifiers it will actually just include the entire license text because that's super helpful it makes your SPDX quite large but at least you still have the license again we can do home page, URL CVEs we know all of that we can list all the source files with their checksums that we downloaded and used in order to do the builds that's very helpful we do scan the source files for the SPDX license identifier and we'll include that information from all the source files that we include in the SBOM similarly with the packages we know all the files in the packages with their checksums we can also do package file generated from using the debug data so this is something that's extremely powerful and basically what it means is we'll look at the debug data which we always generate unless someone tries really hard to turn it off when we compile a binary we we generate the debug data and we'll actually look through it and trace back the source code files referenced in the debug data to the original recipes that generated them which might not be the recipe that you're currently building where this becomes super powerful is with when you're including static libraries because what this allows us to do is trace back binaries that are including static libraries to the recipes that originally produced those static libraries which traditionally is very difficult to do to know if your binary is using a static library in the first place and then also to trace it back to the original source it's not perfect using the debug data but it's you know it's pretty good again we also have all the build time dependencies and the run time dependencies and we can also generate a source code archive if you want to do additional analysis of the source code with phosology or something like that okay so what can we generate spdx for and the short answer is basically anything we can build we can generate some amount of spdx for because it just kind of turns on and happens so all of your on target your cc++ for train your sort of traditional languages if you want to call it that we can very easily generate lots of spdx again all those native tools we can generate spdx for we can also importantly generate spdx for the linux kernel as far as I know we are one of the few projects that can generate meaningful spdx for the linux kernel a lot of some work went into that and it's really awesome that we can do that target images again anything that we classify as a target image we can produce spdx for stks container images so that's a good one so if you want to know how do I build a container image where I have spdx from the build of that container you can try building it with the octa project and try that out so that's really cool and the same with VM images right and I have Rust and Go under construction mostly just because the the core thing that you're trying to build in Rust and Go like what the recipe is actually written for we have really good spdx for that it's just basically the same as most of the other recipes the part that's missing is Rust and Go have their own package managers so getting the spdx from the cargo crates that we pull down is a little more difficult and I don't think that's working I didn't actually try it I should have but so I don't know if that's quite working yet and the same with Go right it pulls in Go modules I don't know what they're called it pulls in the Go stuff I'm not a Rust or Go person so if you are and you want to figure out how to get this to work so that we can actually get the spdx from those crates and things like you know that'd be awesome I would be super excited about that there's a couple of configuration knobs you can set to control the basically amount or style of spdx output you get spdx includes sources as a knob you can turn on and off so this is what actually includes the list of all the source files and their relationships in the generated spdx document and the license information in them it's off by default because it's just huge output the test that I ran the root file system I generated uncompressed was 20 megabytes the spdx compressed was 23 megabytes so you know more spdx documentation than the actual thing you produced when you turn this on so yeah it's big spdx archive sources that's the thing that gives you the tar ball that has all the sources that you can pass to phosology or whatever actually archive packaged will give you the packaged files if you want to do additional analysis on them for some reason those are both off by default just because they take time and then the one that I added just last week because I got tired of looking at the single line of JSON output we produced to keep the output small is spdx pretty that'll give you the nice new lines and indentation if you're manually looking through these things because it can get a little tankers after a while you can publish your spdx results on the internet I'm not going to talk about this at all in time it's there I will publish my slides after this alright so what's actually in our spdx yeah so when you do a build with spdx with the create spdx class enabled you get some output in your deploy directory if you're not familiar with the octo output don't worry about this too much it'll make sense in typical fashion these are actually sim links to the actual time link time stamp version files I'm just going to pretend they're the actual files because it's just easier so there's three files that we really care about here this first file is the spdx file for the image itself so this is the spdx file that's going to say this is what went into this image and basically what it has is a whole bunch of external document references in each package that got installed in that image the second file this is the one that you really are most interested in this is the compressed tarball that contains all the spdx documents for the image so it starts with that top level image spdx document that I was talking about and then it recursively follows all of the external document references it finds starting with that file and pulling in those documents and then pulling in the references to those documents until it runs out of things to pull into the tarball and that is the compendium of all the spdx documents that are relevant for this image and then this last file is a JSON index file that we create this isn't an spdx thing this is just something we came up with so spdx documents are referenced by their document namespace which is like a uid-ish type thing and so that's how you when you reference one document from another you reference it by that this file lets you easily map the document namespace to the file name because our file names aren't named by the document namespace just for reasons so you can use this if you're trying to transverse the large number of documents that we have to find the file names that you're looking for so if we do a listing of this spdx archive we'll see the files that are in it and we can dig into that and see what's in here so this is, again this is that top level image spdx file that you saw in the previous directory and this is the index file that's always the last thing in there the interesting files are these ones now so utllinux lsblock and utllinux unshare.spdx.json these are the spdx files that describe the packages we produced these are going to include the binaries and the files and stuff that got installed onto the image that was produced and it's going to describe, you know, it's going to have the check sums and all the file listing and all there due to some quirks in the way that things are generated we actually have a separate document that describes the runtime relationship between packages and I'll get into that a little bit more later so that's in this file so these files are going to have spdx relationships that describe how packages runtime depend on each other so you can use these to track through the runtime dependencies and figure out, like, you know what libraries are being included and what not at runtime and then the final spdx document is for the actual recipe itself so, you know, one recipe might produce several packages in this case the utllinux recipe itself produces the lsblock and shares packages separately but they both then have a relationship back to the original recipe so the recipe basically describes the source code that's got all your cve tracking in it if you've got that source file listing enabled it's going to list all of the things it's got all the licensing information in it it's basically the things that are common to all the packages that got produced so you can think of it like the packages that describe what's on the root file system the recipe describes how those things were built more or less all right, so as you might have guessed we have a lot of relationships in our spdx documents a lot of external document reference relationships specifically in our spdx documents so this is kind of a chart of what that looks like so if we start up here in the upper right corner that's, again, our top level image spdx and we can track down here what it contains the packages that got installed in that image and so then those packages, again, contain the individual package files that are in them and then if we go down from there we can see that these have a generated from relationship to the recipe that produced them and that recipe again has this contains relationship on its source code and also then this is how we track the build time dependencies between recipes so build time dependencies are a property of recipes whereas run time dependencies are a property of packages so if we go back up here for our package spdx we can see we'll also get these generated from on other recipes which weren't the ones that produced us and that's that tracking through the debug source for static libraries and stuff like that and then if we move up here we've got these run time documents so as it turns out package managers don't require run time dependencies to be an acyclic graph it's perfectly acceptable to have cyclical run time dependencies it just means you have to install all of those packages together in one big lump which is fine however when you are referencing spdx documents you reference them by their document namespace and their checksum which means once you've written an spdx document you can't modify it which is really good for supply chain tracking but it's really bad when you can have circular run time dependencies because it means you can't put the run time dependencies in the package because there's not a single node you can start at in the graph that will give you a directed acyclic graph that you can walk through to write out all of your dependencies so because of that we actually have to write these run time dependencies in a separate document so that the package spdx's that are being referenced have all been written and finalized so that we can reference their checksum sort of a quirk of the way spdx currently works so we have these run time dependencies and these basically just define the run time dependencies between two package spdx documents they're really not that exciting they just have a bunch of run time dependency of relationships in them so future things that we'd like to improve with this some of this looks a little weird I won't say that it isn't I think we're one of the first projects that's really generated this level of spdx output and so that's really good and so we definitely found some edge cases and things in sort of the spdx spec and so we're working very closely with the spdx project and it's good for both of us it's good that we have a use case that's producing these things and finding them and it's good for us to help them find these use cases and we're going to be able to help them come up with a spec that helps us too so this is all really good I'm super excited about that we'd really like to be able to pull in spdx and sbombs from other upstream source code particularly like projects using reuse or something like that we'd like to be able to pull in their sbombs I don't think we would necessarily replace what we're producing with that but we could definitely generate external document references to point to that and pull that into that entire archive and I think that would work really well because then you can see exactly like upstream said this, we said this it's just one more link in that software supply chain which would be awesome we definitely like to include more spdx fields we have a whole bunch of information about how we built the source code that we aren't currently describing at all we have the cflags, the linker flags all of those things we could include we could even include the shell script we actually ran to do the build it's crazy stuff like that all really useful for supply chain reasons okay so moving on to more supply chain related things we've got reproducible builds so I'm going to talk about how the project can help you have reproducible builds in the things that you build so why do we need reproducible builds so there's a bunch of reasons so we need to be able to know what like in order to be able to resist attack we need to know if something is worth looking at to see if it's been compromised and the best way to do that is to have it build reproducibly because then it's very easy to say oh this change that shouldn't have happened if you want to be able to trust your compiler there's techniques for doing that like a diverse double compilation but in order to do that you need your builds to be reproducible there's also quality assurance reasons like the code or the things that we're shipping to our customers we want to be able to reproduce on our desk so that we don't get weird race conditions that only show up in the field and things like that so we want reproducible builds for that reason also smaller binary differences means smaller updates if you're doing delta updates and increased development speed you know if something hasn't changed it doesn't need to be rebuilt and that would be awesome I highly recommend you go check out reproduciblebuilds.org they have a whole lot of information on reproducible builds that's awesome just from like a project standpoint we really like the idea of reproducible builds because when we create these task caches what we're really saying is like we're expecting this output from the recipe and really ideally there should be a one-to-one correspondence between the task cache that we create and the binary output that we generate so just kind of from a intrinsic project perspective we really like reproducible builds for that reason too so how are we ensuring that builds are reproducible so the Yachto Autobilder does a regular tests for regressions in reproducible builds and this is really awesome you can see the results there on that link if you would like we're currently testing about 11,000 target packages we're not currently directly checking the native builds for reproducibility directly we're sort of indirectly testing them by ensuring that they don't change the target output obviously and we're doing this across the three different package formats that we support IPK Debian and RPM we also do this build across multiple build hosts so we'll do builds on Fedora Ubuntu, CentOS Debian and then compare the results between them and see if they change so this is even ensuring that if you switch build hosts you'll still get the same output there and we have tooling in place to do automatic Difascope output Difascope HTML output if we find something that isn't reproducible so this it makes some things very easy to debug I'm not going to say diagnosing and debugging reproducibility problems is easy but this can help with some of them if you want reproducible builds you do need to test your own it's great that upstream is doing reproducibility testing and making sure that packages are generally reproducible but if you're not testing it in your actual images you don't know for sure if they're reproducible or not fortunately it's actually quite easy to test this you can write this three line python file right here in your own layer and basically just replace this my image with whatever image you want to test for reproducibility and then you can run this command down here and that will run the reproducibility test go get lunch or go to bed it takes a really long time alright so build tools tar ball this thing is exciting so this gives you s bombs all the way down so this is that build tools tar ball that I was talking about so there's actually two different build tools tar ball there's the build tools tar ball which is sort of a minimal set of tools that we use in order to sort of paper over some of the host differences between builds and then there's a bigger version of the build tools tar ball called build tools tar ball extended which basically includes every host tool that you need to build so build tools tar ball and build tools tar ball extended are sdks at the end of the day they're just sdks but they're designed to replace your host tools and so if you use the build tools tar ball extended which is the one I'm going to talk about now you can use it to replace all of your host tools and what that means is that all the things that I've just talked about with reproducibility and s bombs and all of that stuff can now apply to that last little bit of host tools that are obviously covered so if you do this you can now trace your target image supply chain all the way back through your target packages to your native tools and cross compilers and even into your host tools if you want to get like really crazy you could have an air gap to build host where you go through the I won't call it not painful effort of building build tools tar ball extended on that air gap host build tools tar ball on your CI and developer systems they could use that to build all of their stuff and then when they build a final target image you would be able to trace that all the way back to the air gap system in your supply chain so super powerful really awesome if you want that really deep supply chain alright special thanks to Saul Wald I did a lot of the generic spdx generation support and he did the stuff that was necessary to make the Linux kernel have spdx support which is really awesome Ross Burton did a bunch of licensing work to make all our licenses use spdx identifiers which is awesome Andres did the SDK support which is the thing that allows the build tools tar ball to do what it does and Richard Purdy just does a whole bunch of stuff with the project and lots of other people that contributed to the project I probably missed a bunch of people yes if you'd like to get involved with what we're doing here we are on IRC and Matrix I think I think it's Matrix there's a bridge you can find us at these channels there's also a weekly technical meeting that you can attend where we call in and talk about technical project stuff there's also a weekly bug triage meeting where we go over all the bugs that have been submitted the last week and things like that there's also an open embedded happy hour last Wednesday of every month the next one is next Wednesday and there is a yacht project summit which is twice yearly and I'm sorry I couldn't get a link for that they didn't really have a generic link yet alright questions oh how do we generate spdx for dynamically generated kernel modules dynamically loaded kernel modules so I believe the way that that would work is they're basically treated so they're going to have a package that gets installed on the root file system so you're going to get a package of spdx for them and then you'll be able to trace that back to the recipe and that recipe should include all the source code for that kernel module a lot of the stuff is all just it's all very similar our recipes are all operate they do different things in different steps but they're kind of all doing very similar things yeah that's how I suspect it will work I haven't actually tested that I could any other questions how much time we got two minutes okay sorry can you say that again oh how do you take all of the spdx output that we generate and distill it for your company lawyer so like I said we're a fairly early provider of this level of information and so I'm positive the tooling will catch up with us oh Kate's got oh okay so apparently you can translate the JSON to spreadsheets which I hear lawyers like so you can do that it is a lot of output and yeah I think we're a little bit ahead of the tooling at this point but I'm positive it'll catch up yeah I'm like I said we're working very closely with the spdx project to get some of the tooling I think oh thank you um yes this the question was when was this released um I know it's it's definitely in 4.0 um I should have checked on the timeline this actually went in like a while ago so it's probably I'm sorry say that again yeah yeah the the past release so the 3.2 I don't remember I don't know if the I don't know if the previous LTS dunfell 3.1 has it um I'd have to check but if if it doesn't the release after it does it's also um it's it's not really that complicated it's just one file um so it's potentially something that could be easily back ported like you could just you could probably just copy if it's not in whatever release you need you might just be able to copy the bb class honestly um it's really not that much to it and I see we're done so any other questions feel free to come talk to me I could talk about spdx for probably hours