Well, I think, yes, I think it's already clear that we've entered the greatest crisis in capitalism's history. Like Alan mentioned last night, the IMF said that this would be the worst crisis since the Great Depression. And I think that really is very much an underestimation. The Bank of England are far less optimistic. You know, they say that this would be the worst crisis for at least 300 years. But I think in reality, none of them have any idea. There's no power really for the period that we've entered into. And Mark Twain said that history doesn't repeat itself, but it does rhyme. I think there's a truth to that. Because if you can draw out the processes taking place in society, you can understand them. It does help guide our perspectives for the future. And I think it's really worth discussing this period that of the 1930s in the Depression. Since really, I think most people's consciousness today has been really shaped by a long period of capitalist upswing. In particular, the post war boom, but even the kind of period after that, even if people didn't live through it themselves. Now, of course, there have been serious crises since the 30s. There was a world crisis in the 1970s. Obviously 2008 and since then, there have been numerous smaller recessions. But many people I think still believe that these are just kind of temporary blips and that conditions will return ultimately to those that we had in the 50s and the 60s. The conditions where there was full employment, strong welfare states. In other words, that we can return to capitalism that could be made to work in the interests of working class people. And I think that's false. But I think it's worth looking back to what actually happened before the post war boom with the conditions of the Wall Street crash and the Great Depression. It's really a reminder to us of what capitalism actually looks like when it's in a deep crisis. And I think although the current crisis will indeed look very different in many respects, I do think that the period we're entering into have much more in common with the conditions of the 30s than of the post war boom or more recently. And therefore, you know, the key things I really want to draw out from this talk is what effects did the crisis have on the consciousness of millions of workers. And importantly, what effect it had on the class struggle. And therefore kind of use as a guide to what changes we might expect to see over the coming period ahead. I'll start with a brief summary of what actually happened in October 1929 with the Wall Street crash itself. It's gone down in history as Sarah said this kind of legendary crisis of capitalism that everyone's kind of taught about, at least in a superficial way. And particularly in terms of the dramatic crash that took place on the stock exchange and also the depth of the depression that followed. Now the crash actually began. There was actually downward drift of share prices on the stock exchange during the preceding months of September and October. But the real slump actually began on Friday the 18th of October. It's worth pointing out that, you know, a crash is not just a one act drama doesn't just take place over a few hours or a day or two. It's actually unfolded over weeks. And the decline got worse on the Monday after that weekend. And it culminated in what was referred to as Black Thursday on the 24th of October, which was actually 91 years ago yesterday. That's when the real panic set in with, you know, these chaotic scenes of Wall Street, the huge crowds gathering outside on the streets, these wild rumors being circulated around. And there was the phenomenon where some shares on the market couldn't actually be sold at all for any price. You know, you had what was referred to as black holes appearing in the market. Now, in the afternoon of that Thursday, the things were actually step kind of temporarily stabilized. And that's because there was an emergency meeting of bankers, you know, all the kind of top Wall Street bankers came together. And there was a scratch in the head saying, well, we've got to do something. And they kind of made a public announcement saying they'll do whatever it takes to support the market. And it's, you know, they started buying up shares and so on. And this, this tended to calm things down. But that calm was only kind of a brief respite as the slump actually restarted next Monday on the 28th. As it became clear, became more and more public knowledge that actually the banks weren't actually supporting the market. And actually they were now secretly trying to frantically kind of save themselves and offload their positions. And this culminated the next day on the kind of legendary Black Tuesday, where the fall in one day, he could the gains of all the previous 12 months. And in just the first half hour of trading, 33 million shares were sold and prices fell by 13%. Now, like a typical days trading would see I think between three to four million shares sold. It was considered drastic at five million. The decline continued until the 13th of November, by which point the stock exchange index was 50% below its September high. Now the government of course did try to intervene. They did what they could by cutting taxes. They also cut the interest rate. And this apparently will appear to initially stem the tide. So that from the start of 1930 to April, the market did actually recover by the 44%. And it's worth pointing out that that's actually quite similar to the situation that we see today, where actually the stock exchanges of the world crashed dramatically in March of this year. And since then they've actually gone back to just above the previous peak after enormous government support, at least that's the case with the New York Stock Exchange. So leading economists and kind of catalysts at the time, they were kind of rushing to kind of reassure everything, everyone that everything was fine, that you know the clash was just a blip that actually the real economy was was was absolutely fine, not to panic and so on. And Andrew Mellon, who was the secretary of the US Treasury, he said at the time, the US economy is fundamentally sound. And I think it's worth bearing in mind that we've seen no shortage of politicians over the past few months saying effectively the same today that actually know that everything's just a kind of, once we've got a vaccine and once, you know this, and the COVID situations under control, everything will just go back to normal. The economy will be fundamentally sound. But actually what happened in June of 1930, the fall then resumed. And it did say to reach a new low in December, well below the peak of 1929. Now there are a few ups and downs, but the overall picture was a steady decline until the market reached rock bottom in July of 1932. By which point it settled at 89% down from the record high of September back in 29. Well, things stagnated until early 1933, after which point there was a kind of an uptick on the stock exchange, but it was extremely shallow, and it remained so until 1937. Again after which point again collapsed. And it's worth pointing out that the stock exchange, which is only really kind of not very good kind of reflection of the underlying strength of the economy. I would actually reclaim its 1929 highs until September of 1954, which I think shows two things. Firstly, how deep the rest of the depression actually was during the 30s. But also secondly, how overinflated the stock market was in September of 29. Now the question that's been asked many times is, why did the crash occur. There's been various explanations, I think ranging from the psychological in that there was just a mass panic that gripped the minds of investors to a combination of secondary factors. And in his famous book on the Wall Street crash, John Grau Bray highlights the following factors, he points to the dodgy corporate structures in the USA at the time, the use of what they called leveraged holding companies and investment trusts. In a nutshell these had the effect of multiplying gains when the market was rising, but that effect work both ways when the market crashed the losses were multiplied. There's the banking structure. There was two major New York banks, actually thousands of other smaller banks across the USA. There was no federal insurance and deposits. So you know, one bankruptcy could easily start at one on all the other banks. And that's one of the factors of what they called trading on the margin, which really meant buying shares and credit. So for example, you might bear it by shares price to 1000 pounds, but with only just 200 pounds of your own cash, you borrow the extra 800 pounds from a bank. Now that worked fine when the market was rising into sometimes you saw shares rising by kind of 1015 20% even in just one day, such as the kind of frenzy. But you know when falling, obviously the banks then called in the loans, many of these loans were then unpayable and that had had huge repercussions. There was also the unsound balance of foreign trade. So the USA become accredited nation following World War One. After the crash clearly that credit then dried up. So the bank is of course demanding payments from debtor countries in gold, of course these countries, these debts couldn't be paid. So the only option for these debtor countries to eliminate their own deficits was even exporting more of their own commodities to USA. And that was of course then blocked by tariffs, or by reducing their imports, which then cut the market for American goods. And then finally there's also the unequal distribution of the income. So I think in 1929 the richest 5% of the US population received 33% of the wealth. So most of the demand in the economy had to come from the rich, even if you kind of, you know, investment or kind of luxury spending. And when that dried up of course had a huge impact. Now, it's worth pointing out that these secondary factors certainly had an influence and why the crash was so deep and why it turned into a depression. I don't think really they explain why this took place in the first place. It's like when you hear the explanation of the so-called credit crunch of 2007 as being the cause for the 2008 crash. You know, really, the crash was a product of all the accumulated contradictions of the previous period, culminating in what we call as Marxist crisis of overproduction. You can see this. So from 1900 to 1929, there had in fact been a colossal development of the productive forces in the USA. The total wealth of the country increased from $86 billion to $361 billion. So that's over four times in just those three decades. From 1919 to 1929 the productivity of industry rose from between 40 to 50%. But at the same time wages and prices remain stable, which had the effect of rapidly rising profits for those at the top, and also an explosion of inequality. Now, the richest 0.1% of the population had the same income as the poorest 43%. So of course, what were they going to do with all that money? Well, of course, they're going to try and make more money, of course. There was this kind of orgy of speculation on the stock exchange. And by even 1926, it was clear that there was a feverish speculative element to the boom. Profits as well as the stock market were booming. Industrial profits rose 156% in just the five years before 29. But in that same period, industrial share prices trebled. And that was really indicative of this massive rise in fictitious capital that was taking place. As soon as it's becoming saturated worldwide, the ruling class could clearly get richer just through speculation on the stock exchange and through actually investing their wealth back into production. Now the system was already reaching its limits by late 1928 to early 1929. And again, this ties back to what I was saying about it being, you know, a classic case of overproduction, which really is inherent in the dynamics of capitalism, since production is only for profit. Now, as Mark explained, the working class produced all value in the form of commodities, but they only paid a portion of that value in the form of wages. But of course, for capitalists to realize a profit, they have to sell the commodities that are being produced by the workers in their factories and workplaces and so on. The problem is how can they do this if the working class as a whole only receives a fraction of the total value that they're producing. As economists put it, the problem is where will the effective demand in the economy come from, in order for all these goods to be sold. Now as Marxists, we understand that there's three main methods that the ruling class can use to overcome this contradiction. Otherwise, you could say, well, why isn't the system permanently in crisis. You can see how all of these were reaching their limits in the late 1920s. So firstly, you had the expansion of world trade. Now remember this was a time when most of the world was directly colonized by a few imperialist powers. The USA had actually expanded into areas of Latin America. And if not kind of directly colonizing these countries, they effectively were colonies, places like Cuba, Nicaragua, also the Philippines as well. Of course, there are limits to expansion when the kind of the whole world is already being carved up. And secondly, there's what Marxist term investment in department two goods, which means kind of more means of production, essentially, things that aren't going to be consumed by the working class, but things that kind of to be invested in by capitalists, things like more machinery, building these factories, actually more materials, infrastructure and so on. But actually investment was slowing down in the 1920s. It was becoming increasingly saturated. And therefore, you know, it wasn't even possible to use the existing industrial capacity and make a profit. There was I think, industrial capacity was only about 80% used at this time. So if you couldn't already use what you had invested, why would you invest in building more capacity. And then thirdly, there was the use of credit. So massive amounts were being lent to other countries so that they could buy American goods. There was expansion of credit domestically. So, you know, people could afford to buy cars or other big ticket items by paying for them on payment plans. There's also, as mentioned, this huge expansion of credit so that investors could buy shares on the stock exchange without putting up the full value. But all of these things have a limit, of course, and you could see the economy was starting to slow down by mid 1929. Industrial production in the USA actually fell from an index of 127 in June of 29 to 122 in September and 117 in October. And remember, this was at a time when leading economists and politicians were saying, actually, the economy is fine, you know, the crash was an accident. But really you can see from these figures that the decline had already set in. And I think what quite stark is the figures for car production, which declined from 660,000 units in March of 29 to 319,000 by October, so more than halved in just seven months. Now, after the crash took place in late October, those figures then collapsed even more dramatically. But it kind of shows how things were already going downwards. Now, there were various incidents cited as the cause of the bubble ultimately bursting. There was an unfavorable ruling against utility company in Massachusetts, and it was Boston Edison, which among other things said that the shares have been massively overvalued. There's also the collapse of the Clarence Hartley financial group in Britain, which came as a huge shock to investors really spooked a lot of people. But whatever it was, it was merely the accident that kind of allowed for this deep necessity of the bubble that had to burst to express itself. Now what happened then on the stock exchange, you know, was really a surface feature of a far deeper process of this crisis of overproduction. Some contradictions are built up to tipping point, whereby it took, you know, once no incident or incidents to really transform the whole quality of the situation from a beam into a slum. And it's similar to a forest fire, whereby it takes a long period for the forest to become dried out, but at a certain point all it takes is a small spark to set the whole thing ablaze. Now what were the consequences? In the USA, it gave rise to conditions known as the Great Depression, and the crisis on Wall Street then quickly rippled through to the rest of the economy. And as we say, like cause becomes effect, and effect becomes cause, the whole thing entered into a downward spiral, and all those secondary factors that I outlined earlier then came into play. And the collapse of share prices meant that loans taken out to buy shares on the margin were then quickly called in. And you can see how credit, which was used previously to expand the boom, and now turned into its opposite. And instead you had these unpayable debts which now had to be repaid, and a wave of defaults then led to a crisis of the whole banking system. And I said with no federal insurance and bank deposits, the collapse of one bank meant losing your entire savings. So hence runs on banks became very commonplace, so that between 1929 and 1933, over 9000 US banks collapsed. Now even before the crash, companies were cutting production, as I said the market was becoming saturated. But now as both demand and credit dried up, so did investment and production. Again, this is the thing of cause becoming an effect, an effect becoming cause. You enter this vicious downward spiral whereby millions of workers were being made unemployed, as they could no longer be profitably exploited by the ruling class. That then led to a vicious circle of a further collapse in demand, and a wave of corporate bankruptcies. Now you can see this in the figures, the Federal Reserve Board of Industrial Production, the index declined by almost 50%, between 1929 to 1932. Now the crisis of overproduction was really graphically illustrated by the figures for capacity utilization. In 1920 it was 94%. And then averaged about 84% over the decade of the 20s, the so-called Royal Ring 20s. But by 1930 it had fallen to 66%, and it reached a lower 42% in 1932. Now in July of that year, steel operations in the USA were only functioning at 12% of their capacity, which I think very graphically shows the depth of the crisis. And the only way for the ruling class to eliminate this so-called excess capacity was really to close down factories and massively lower prices. Now that of course introduced deflation into the economy, since the unemployed couldn't afford to spend. And those that could still afford to buy things were very reluctant to spend on the big ticker items today, knowing that prices would be cheaper in the future. And that deflation then also meant that servicing debts became relatively more expensive, and that further dragged down the economy. Now the effects of the crash, as with any crisis under capitalism, was of course the working class and the poorest who were made to fit the bill. And you saw a massive attack on the already low living standards of workers and poor farmers. And therefore the whole, the economic crisis was transferred into a huge social crisis. Unemployment of course skyrocketed. So in 1929 a significant 1.5 million people were already unemployed. That's about 3% of the work for us. But that then leapt to more than 12 million by 1932 and then estimated 13 to 15 million by March of 1933. That was about 25% of all workers, so 37% if you exclude farm workers. And overall it's thought that 34 million Americans belong to families with no regular full-time wage earner. And therefore with no federal system of social security, workers were forced to turn to what limited charitable relief existed or otherwise face starvation. So, you know, unable to pay the rent, millions of people ended up homeless or travelling the country in search of work. Hundreds of thousands ended up living in what became known as Hooverville's, which really were kind of shanty towns built on either derelict land or rubbish dumps and places like this. And in the context of those conditions, the bosses sought to restore profitability by driving down wages and increasing hours. So you had the phenomenon of sweatshops appearing everywhere. You had starvation wages were common, as was child labour, and many workers were forced to work 60 or 70 hours a week or more. Now with the shock of the crisis and with the threat of destitution and really the lack of leadership coming from the trade unions and the workers' parties, the bosses were largely successful in these attacks. Now just to speak a bit about the global consequences, obviously the crisis in the USA was very quickly to become a worldwide crisis. And that's because in 1929 there were just four countries in the world that contributed up to 70% of world GDP. That was the USA, Britain, Germany and France. And they also kind of a dramatic collapse in the production as the system was so interconnected. Now in June 1930 the US introduced the Smoot-Hawley-Tariff Act and that was to try and protect US agricultural capitalists in particular. And that had a devastating impact on the agricultural economies of central Europe as now the main market for their exports was now closed off. And places like this have borrowed heavily from French and British banks, as had a lot of German industry. And of course when those debts were then defaulted on, it drove Britain and France into financial crisis. And that then rippled over back into US banks, as I said, the whole system was interconnected. Now eventually Britain was forced off the gold standard and was forced to devalue its currency, but soon every country had to follow suit. All the ruling classes were trying to maintain the share of a declining world market through competitive devaluations with the imposition of tariffs and quotas, in other words through a trade war. And therefore US imports and exports declined by 70% between 29 to 32. And actually world trade in 1933 was less than a third of its 29 level. Now it's worth pointing out that I think the collapse of world trade was inevitable due to this unavoidable slump in production and investment. But the policies of protectionism and devaluation, therefore kind of inevitably followed, but they had the effect of exacerbating the slump. They helped turn it from a recession into a deep depression. And I think, you know, whilst today a lot of the kind of capitalist strategists are warning against a return to protectionism. Really, I think they can warn all they like, but it's really built into the logic of competition between a scramble for a share of a declining world market. Each national ruling class is going to try and pass the burden of the crisis on to others. Now I want to talk a bit about the new deal in the United States is by 1933, you know the world was clearly in the midst of a very severe depression. And from the standpoint of the ruling class conditions in the USA were actually getting critical. Now, Herbert Hoover, who was the president at the time of the crash, he generally followed this kind of laissez-faire approach to handling the crisis. He said the state really shouldn't intervene. And therefore, although taxes and interest rates were lowered to try and encourage investments, since avenues for profitable investment were so small and actually in decline, these measures ended up having little effect. Now, Herbert Hoover's commitment to balanced budgets meant that government expenditure during this period was actually cut back. And since tax receipts so collapsed, the government spending therefore also had to be cut. And unlike today, the banks were actually allowed to collapse, wiping out deposited savings in the process. And with no welfare, the ground was being prepared for social explosion. Now for millions of people, the irrationality of the system was very clear to see. On the one hand, millions of tons of food around, but it wasn't profitable to transport it or to sell it. And therefore it was left to rot in the fields. You had warehouses literally full of clothing, but people couldn't afford it. So, you know, they're walking around barefoot wearing rags and so on. There was lots of empty houses, but you know, had people living on the streets or in these Hooverville shanty towns. And therefore, you know, people began more and more to get organized. You had unemployed councils established all over the country, typically led by communists. And they organized people to resist evictions as well as to pressure the relief commission to try and ensure families obtained aid. More alarmingly for the ruling class with a number of clashes with the police by the poor taking matters into their own hands. So actually from 1931 onwards, you know, the phenomenon of hundreds or sometimes even thousands of unemployed workers with storm factories demanding work from the bosses or storming government buildings to demand food and shelter. You also had farmers were being thrown off the land in their thousands as the banks before closing the mortgages that led to widespread violence and rebellion in the countryside. And then the communities more and more began to organize to fight off the bailiffs and the police when they were trying to carry out evictions. I remember that this was a population that was largely armed. All you had that they were blockading rows to try and stop cheap produce from elsewhere from reaching the towns and undercutting the local farmers. So in January of 1933, he had as Edward O'Neill, who was the head of the Farm Bureau Federation. He warned a Senate committee. He said that unless something is done for the American farmer will have a revolution in the countryside within less than 12 months. You also had increasingly workers began to organize self help groups to try and bypass the restrictions of the market. And one very notable example is in the coal region of Pennsylvania, where tens of thousands of unemployed miners began digging small pits on company property. They were trucking bootleg coal into the cities and selling it below market price. Now when the police tried to prosecute workers involved in this, actually no jury would convict them and it showed this kind of basic class solidarity. And this was very dangerous from the point of view of the ruling class, you know, the fact of workers beginning to organize production and distribution themselves and kind of bypassing the capitalist market and overriding capitalist private property. Now at the same time the ruling class were increasingly losing confidence in their own ability to rule. And there was one Rexwood Tugwell, who was one of the architects of the New Deal, he summed up the situation. And he said, I do not think it is too much to say that on March the fourth, which was the day of the general election, we were confronted with a choice between an orderly revolution, a peaceful and rapid departure from past concepts, and a violent and disorderly overthrow of the whole capitalist structure. But that really was kind of behind the election of Roosevelt in 1933. When he came to power he had really no alternative but to act. And therefore he reorganized the banking system which by that point had almost completely collapsed. But he also introduced the New Deal. This is because there's a lot of myths about the New Deal. And you often hear today, you know, calls for a new New Deal, typically a green New Deal, but the reality of it was that it was really a program of reform legislation. There was an attempt to rescue capitalism from itself. And they said that work fair schemes in employing millions of workers on construction and conservation projects, although not on full wages. They provided billions of dollars to fund over hundreds of thousands of construction projects. It's thought that about 80% of all public construction in the USA from 33 to 37 was funded by the government. They also passed the National Industrial Recovery Act, which really was to try and prevent a race to the bottom by the capitalist. And this was because by this point wages had really been driven down to starvation levels. And of course it was rational for each bus to do this in order to try and increase their own profits.  But the overall effects of that was to reduce the overall effect of demand in the economy, as you know who is then going to be able to buy what is actually being produced. So the government had to kind of step in to effectively suspend anti trust laws, and actually brought leading capitalists of each industry together to draft what they call a code of fair competition. The idea was to try and regulate their sector, and they would set minimum wages, hours, prices, they put quotas and output and so on. And it was effectively giving monopoly powers over whole industries to a layer of capitalists in order to try and boost profitability. And really it was a tacit admission that the market had ultimately failed, and that they needed the element of planning to get the economy going again. But no, it was not planning in a socialist way, by the working class in order to meet the needs of society, but planning by the bosses in order to boost their own profitability. Now, Roosevelt also dealt with overproduction by of course destroying the productive forces and therefore he gave subsidies to farmers to not produce. And that's because farm prices had collapsed by over 50% between 29 and 32. And in a show, despite people starving, there was still massive overproduction from the standpoint of what could be sold profitably on the market.  So in spring of 1933, the government set up the agricultural adjustment agency to try and administer subsidies. And one was, though, that millions of acres of crops had already been planted, and millions of animals had already been born. And therefore farmers were then paid to destroy 10 million acres of cotton, which was a quarter of that year's crop, and slaughter 6 million piglets. Now this in particular cause enormous uproar. So you had Henry Wallace, who was the Secretary for Agriculture at the time. He was publicly trying to reassure people. He explained, you know, don't worry, this is just simply the logic of capitalism. So there's no need to panic. There's no, no one should be angry about this. And he argued falsely that nobody had been morally indignant when American industry had reduced its output by about $20 billion worth of goods over the previous years. But in doing so, really, he just kind of shot a light onto the real workings of capitalism. And it was clear that the whole irrationality of the system was very evident for everyone. Now, over the course of the New Deal, various other reforms were carried out in order to try and stabilize the system. These on banks were put to an end through a federal insurance scheme and deposits. Debt relief was provided to mortgage holders by means of government payments to the banks. And not only did this prevent thousands of foreclosures, but it also helped prop up the failing banks, which were otherwise riddled with bad debts. I think you can see today, you know why banks are so keen to give mortgage holidays, because the banks are on the one hand making a killing on the extra interest that they receive from this. But otherwise, they were to face an enormous mountain of bad debt from the thousands of people who just simply weren't able to keep up their payments. And for the first time in US history, there was a nationwide system of unemployment benefits and old age pensions were introduced. Now, all of that in total meant a sharp break of these previous policies of laissez faire capitalism, whereby the state mustn't intervene. And it was really a tacit acknowledgement that if left to the mercy of the market, the resulting destitution would ultimately threaten the very viability of the capitalist rule. And therefore what the capitalist could not do as individuals due to competition and the anarchy of the market, the state would step in to try and do for them. And this was really summed up by Roosevelt himself in 1936. He said, nobody in the United States believes more firmly than I do in the system of private business, private property, and private property and private profit. It was this administration which saved the system of private profit and free enterprise after it had been dragged to the brink of ruin. And I think that would do very well for the left reformist today, who celebrate Roosevelt and celebrate the New Deal as a model to try and remember that. Now, of course, these policies did have an effect in alleviating some of the worst effects of the depression. So after reaching rock bottom, American GDP did then rise by 34% between 1933 and 37. But however, for a majority of the working class and the poor in the USA conditions only really went from very bad to just bad. So even at its peak, the public works program only employed a quarter of the total unemployed and unemployment in the 30s never actually fell below 8 million people. The minimum wages during this period are only just enough to cover the bare necessities and those who are on relief, either on work programs or through social security, that even worse. When the economy seemingly back on track by 1937, Roosevelt came under enormous pressure from the capitalist class to return to a policy of ballots budgets. And therefore, when he won the presidential election in 36, he then succumbed to this pressure. And in 1937 he scrapped the relief programs, and he again raised taxes. He succeeded in reducing the federal budget deficit by 1938. But by switching up the life support to the economy, it resulted in a dramatic fall in economic activity. And that's because really the key question of a lack of profitable markets remain. So production did not reach the levels of 29 until 1941 when unemployment was still at 10%. But really it was only the impact of the Second World War that really revived the economy. And that canes himself would later acknowledge, and it was really only through the military kind of mopping up unemployment, most a massive expenditure on military production and so on. But ultimately this, this crisis would be ended. Now, as I said at the start, the important thing for us to understand is the effect on the class struggle. As you know we always say it's great events that transform consciousness on a mass scale. And as this crisis and the collapsing living standards was certainly a huge shock to millions of people, and it would lead to a profound questioning of the system. You didn't immediately see an impact in terms of industrial militancy. That she took a few years for the initial shock and despair of the crisis to wear off, especially when there's such high unemployment and destitution, and most workers probably didn't want to rock the boat. But of course there was a lot of anger building up, but a lot of people felt powerless to do anything. They didn't think to lack of leadership coming from the workers organizations. But when the economy turned up a little from the depths of 1933. A lot of workers then saw their chance to try and fight back. So in 1933 three many three times as many workers went on strike as in the previous year, but the real turning point was in 1934, which was an explosion of industrial militancy. And just to then, most struggles involved mainly the organized workers are striking to improve or rather even defend their wages and hours. But now from 34 it was the right to organize and union recognition that were the main focus of strikes. Now the confidence of workers to organize was in part boosted by the provision for section seven a of this recently passed National Industrial Recovery Act. And this gave at least on papers, workers are quite the right to organize and bargain collectively through representatives of their own choosing. But despite that right existing on paper, it was routinely ignored by the bosses.  So of course the capitalist and this is the support of the police, the courts and gangs of higher thugs to viciously smash strikes during this period. Remember strikes then they weren't like some of the strikes that we've seen recently and that maybe some comrades here have participated in. Now these were extremely militant battles. And often the police would turn up to open fire at pickets, they launched tear gas those sent in fascist gangs. But nevertheless a series of large strikes developed in the spring of 1934. In Minneapolis, which was a key distribution city truck drivers organize a solid strike to demand recognition of the team says union. Now after four months and several violent clashes, and I really do mean violent. There was one incident the police shot 67 strikers and killed two of them. The employers finally conceded. In San Francisco there was a strike of the longshore men, which is dockers, the union recognition, ultimately turned into a general strike in July of 1934, and the employers ended up giving into the union's demands. Elsewhere there was a strike of 325,000 textile workers in the American south that very quickly spread nationwide. So by mid September 421,000 textile workers had joined the strike. Now, many of those strikes in 34, and particularly those in mass production industries did actually end in failure. But most of that was, was due to the conservatism of the leadership of the American Federation of Labor, the AFL, and its policy of organizing workers and craft lines. So for example, when rubber workers in Acron, Ohio, set up a union on a plant basis, the AFL leadership actually intervened to split up the workers into 19 separate craft locals. And such division would of course prove fatal. But however, with pressure from below, we'd actually end up transforming the union structures. In the course of 1934 to 35, hundreds of thousands of workers in mass production industries began to get organized. And of course, the AFL couldn't simply ignore them. So 1935 established a committee for industrial and organization. The idea was to organize workers by industry. So all workers in a single plant would be united together. And with increasing resistance by a layer of the AFL leadership, that committee ended up splitting from the AFL in November of 35 to form the Congress for Industrial Organization, which is known as the CIO. Now the growth of the CIO was actually spectacular. There was clearly a militant mood that built up in the working class, which is desperate to find an expression. And with the AFL providing little lead, what strikes it did organize, you know, clearly under pressure from below, they were characterized by desire by the leadership to kind of compromise with the bosses at any cost. But in contrast, the CIO was characterized by militancy by energy. And importantly, it was getting results. And that rise in militancy was reflected in the changing tactics of striking workers. So in the mid-1930s, strikers began occupying their factories in order to prevent the use of scabs. And this tactic was known as the sit-down strike, and it began to quickly spread. So in 36, there were only 48 sit-down strikes in the whole of the USA. But by the next year, this had risen to 477. And this included significant battles at General Motors and the Chrysler plants, where the workers were successful in winning union recognition. So within four years of its foundation, the CIO organized four million workers, and that included breakthroughs in the auto industry, steel, rubber and packing houses. But even white collar workers and agricultural workers were rushing to join the union. And the general rise in militancy also had the effect of whipping the AFL into shape. So by 1939, it then reported four million workers, which was the highest level since 1920 peak. And therefore the situation was really transformed. And that's something that the left performers today can't imagine. So in the space of four years from 1933 to 37, the total percentage of workers organized in the USA rose from just 8% to 22%. And that was, of course, an enormous step forward and important conquest for the working class, as described as labor's giant leap. Despite the enormous potential that exists to harness that militancy into a political struggle for socialism, union leadership still lag far behind. As ultimately there were reformists, you know, they saw the class struggles being limited to simply the economic struggle. They had no perspective of transforming it into a political struggle for the working class to take power. They ultimately played the role of dampening them, see, dampening the militancy of the working class. And they simply encouraged workers to support the Democratic Party, rather than lead a struggle to create a mass workers party. Now, worldwide the radicalization wasn't just limited to the USA in Britain, you had the split of the ILP from the Labour Party, because you had Labour in power carrying out a strategy against the working class. You had the revolutionary opportunities in France with a wave of sit-down strikes in 36. You had Spain, the revolution beginning in 31, leading to the Civil War starting in 36. But also the black reaction in Nazi Germany. You know, all these things, of course, have many factors behind them. But I think the economic depression was clearly important. Now, I'm running out of time, so I just want to come in by kind of drawing out what I think are the main lessons. I don't stress the limits to any historical analogy. And of course the 30s is not going to look exactly like the period we're in now. But I think, firstly, I would say that, you know, COVID-19 isn't the ultimate cause of this current crisis. The ultimate cause is fundamentally capitalism. I think just imagine things will look very different if the system in the world was not run for profit. Secondly, depression is not a one-act drama, nor does it proceed in kind of a straight line of permanent decline. You know, there can be periods of stabilization, like you saw in early 1930, or even temporary recoveries, like you had in 33 to 37. We mustn't let that disorientate us. You know, it's worth bearing in mind when we see the current recovery on the stock exchange, for example, or economists predicting a V-shaped recession. I think that's very much ruled out. There's massive overproduction that's been built up in the system over years. That's not going to be quickly resolved. Thirdly is evidenced by the New Deal. The ruling class will go to great lengths to save their system when they feel it necessary. Hence the trillions that have been spent by governments worldwide recently in trying to prop up their economies. But all these things ultimately have their limits. And I think we're unlikely to see a return of any kind of significant New Deal or Green New Deal. You know, whilst the ruling class may well be happy for the state to save them, the problem is the colossal debts that have been built up over the previous period in order to try and get out of the previous crisis. So look at in China, for example, 2008 to 2009, they had it in effect the biggest New Deal the world's ever seen. There's that famous statistic that they poured more concrete in China in every three years than the USA did in the whole of the 20th century. But in doing so, they actually quadrupled the state debt so that now the amount of money offered by the government is minuscule in comparison to what came before and also in terms of what is needed. I think more likely you're going to see the prospect of government defaults like sort of Greece, Spain, Ireland, Portugal before or Argentina recently. And that really means massive austerity will be on the edge of the day. Now, fourthly, these events will have a profound impact on consciousness. You know, it'll be the working class and poor who will be made to pay for the crisis. In 1930s, it took a few years for that crisis to be transformed into industrial meditancy. You saw the same thing between 2008 and 2011. But now I think it's likely to be far shorter as that anger has already been built up over the past decade. And actually for many people that the struggles facing them struggles over issues of life and death, you know, kind of, you know, safe operation of work spaces, PPE and so on. And then finally, I think with the rise in the class struggle, we'll see a transformation of the mass organizations of the working class, particularly the trade unions and the political parties. And we of course know that the leadership will play a reactionary role in this in general. You've seen the TUC line up recently with the Tories and the CBI to kind of enact class compromise policies. But there will of course be rising meditancy in the rank and file. And that of course will push a section of the leadership to the left when other cases the leadership will be replaced. In workers' parties, I think they're likely to come even more palerized than they are at the moment. You'll see either the development of a left wing, which I think under these conditions could be pushed very far to the left. Or if that kind of development is blocked, then you see these parties enter into crisis and you see new formations, either as the result of splits or new parties emerging from mass movements. In other words, a continuation of the processes that we've seen over the past decade born in much higher level as the pressure is going to be so much more intense. Now, I think what we could be certain of is that there's going to be no return to kind of so-called new manatee. I think a lot of people still believe that this is a temporary crisis and that once the pandemic is solved, things will go back. But in reality, I think 2020 will be seen really as a kind of fundamental turning point in capitalism's history. Very similar to the way that 1929 was. I think again stress it will be more like the conditions of the 1930s that capitalism will return to. Mass unemployment, destitution, and those of more recent history. And I think it's this process that will fundamentally shake up the consciousness of millions of people around the world. And it's this really that's preparing the ground for social explosion in all countries in which socialism on a world scale will be put firmly back on the agenda. So I'll leave it there. Thank you.


Patriot Prime Reviews is a channel for adult collectors and may not be suitable for children under 13 years of age. Viewer discretion is advised. Hey, what's going on guys? Patriot Prime here once again with another ToyHacks.com sponsored video. But before I get started, I need you guys to do me a favor. If you're watching this video right here and are not a subscriber of Patriot Prime Reviews, please consider hitting that subscribe button right now. It won't cost you a thing, but will help me and my channel out tremendously. Now, on to the decals. And the set we're looking at in this video is for Transformers Shattered Glass Blur. And this decal set includes three decal sheets, one on foil, one on vinyl, and one on clear backing. So now, without further ado, let's take a look at Shattered Glass Blur both before and after ToyHacks decals. And welcome to Patriot Prime Reviews. In robot mode, straight out of the packaging, I think Shattered Glass Blur looks great. I actually prefer this blur over the Studio Series 86 one. Now, taking a closer look at the figure, most of Shattered Glass Blur's paint applications are here on the head sculpt. You can see yellow detailing here on the crest and the side of the helmet. There's silver for the face, evil red eye right there. He's got a black eye patch with some red scarves. More paint applications right there. You can see the red Autobot logo right there under that clear chest piece. Got some paint applications there below the chest, silver paint applications there on the knees. And really, that's about it. But still, this guy looks great. So there you go, guys. There is one final look at Shattered Glass Blur in robot mode before. Did he just move on his own? And after. Shattered Glass Blur looks like a completely new figure now thanks to ToyHacks. Taking a closer look at him, he's got new yellow decals here on the head right there in the middle of the forehead and on either side of his helmet. And he has a brand new metallic looking eye patch. I like that. It looks like he put a steel plate over his missing eye instead of just the black. Now, I do like how the black outline still shows through. Really makes that stand out. He also gets new decals here on top of the chest and a new decal for the translucent window. That was actually like window tint. So just darkened it up instead of completely covering the translucent plastic. And I think that looks great. You can still see the Autobot insignia there on the inside. I like that. ToyHacks, do more of these. Now, on the arms, he gets decals up here all the way around the thruster and new decals on the upper arm that look like Energon fuel lines. So that's really cool. Really breaks up all the blue. On the forearms, he gets new red decals there and a decal here on the wrist. Looks like some kind of button on this side. More red decals and what looks like a readout of the Energon fuel. It's almost like Bane. Like he can activate this button and input more Energon into his body. Here on the crotch plate, we got new gray decals that look really good. I love the looks of those. Really breaks up all that black. Moving on down to the knees, we got new decals there. Lower legs has red decals. They actually look like more Energon fuel lines and there's some decals there on the toes. Now the side of the arms, you're going to notice more red decals up here and there on the forearms, along with chrome lightning effects. So a really new and unique look for Blur. So there you go, guys. Shattered Glass Blur in Robot Mode after. In Vehicle Mode, straight out of the packaging, Shattered Glass Blur looks just as good as he did in Robot Mode. Taking a closer look at the vehicle. The paint applications mainly showing off in this mode is the lightning decals right there all along the front. Those look so good. I mean, they really add to this Vehicle Mode. Then of course he has the translucent windshield right there with the Autobot logo in the center. Silver here on the back and really that's it for the paint applications. But what he has really, really works and I can't wait to see what Toy Hacks does for it. So there you go, guys. Shattered Glass Blur in Vehicle Mode before and after. And Blur's Vehicle Mode looks absolutely insane now. As you can see, the chrome lightning effects go all over the entire vehicle and it looks absolutely bonkers. I love the way the chrome catches the light. That is just too cool. Now these can be a challenge to put on because all of the lightning effects is on that clear, transparent background and it's kind of hard to light them up on the figure due to the fact a lot of them are on black and like I said, they're transparent backgrounds so you have to be very careful to make sure you apply the correct decal in the right spot. But man, oh man, do the end results look amazing. At least in my opinion. The dark decal on the window looks great here. There's also new decals there for the control panel and the seats. There's new red highlights on both sides all along the sides of the vehicle which look really good, really stand out here, here, here and up top along with some there on the back. And speaking of the back, check this out. This is so cool. He's got like red Energon crystals on a see-through engine that helps power him. I dig that. Just, it's so different. And of course you got the decals there on the thrusters and the Energon crystal decals right there on the side as well. This thing just hops now. Also new lightning decals here up on the spoiler and a new yellow decal right there. So, oh, also almost missed the yellow headlights now. I'm still in thrall by all the lightning effects. So there you go guys. There is Shatterglass Blur in vehicle mode. Alright, I'm going to be straight with you guys. This set was a challenge to apply. Mainly because of all of the lightning effect decals or pinstriping, whatever it was, it was all on that translucent clear backing which proved to be quite a chore trying to line up the sticker pieces along with the sculpted details of the figure. I had to take a couple of breaks just to give my eyes a rest. But after I got them all applied, this figure looks amazing now. And in my mind, why Blur looks the way he does is after he took the damage, whatever caused him to lose his eye kind of messed with him mentally and he needed to upgrade himself. So I can only imagine that he got in contact with Shatterglass Ratchet here and he made all of these modifications using the red energon to make himself faster and more powerful. I know the designer of this set watches my videos, so please, in the comments, I want to hear what your inspiration was for this set because I love it. I think you did a fantastic job and really adds to this character and the insanity of the Shatterglass universe. Now guys, if you enjoyed the video, don't forget to like, don't forget to subscribe and don't forget to click that bell icon to get notified when I upload new videos. Once again, guys, this is Patriot Prime, signing out. Ho-la!


I'd like you to welcome Peter Chubb here, who's going to be a talk on SD cards. Good day, folks. This story started about two years ago when we got some Samsung TVs to stick on the kitchen wall to display stuff in our CI system so we could see what the bug rate was, who was winning in terms of the formal methods team and the kernel team as to who's got the most bugs and whether they've actually caught up proving all the code that we've done. Additionally, when we put up the Samsung TVs, the browser in those wasn't adequate. It didn't talk HTML5. So I happened to have one of these things sitting on my desk. We bought the Raspberry Pi some time before to try out for a different project and that project didn't actually get anywhere. So I had a couple on my desk. So I plugged one of these things in. The format was appalling. And I finally traced that to the SD card. And that's where this started. It turns out that there are a lot of embedded systems that I'm interested in where an SD card is either the only or the main storage device. They don't have SATA, they don't have embedded flash. All I've got is an SD card. And that means it's vitally important to get things right. And if you just buy the cheapest SD card from the local post office and take a Leonardo image and DD it on, you're not going to get anywhere.  My first thought was, well, buy a faster card. Maybe we can get something like a, you know, a SanDisk Extreme Pro with GoFaster Stripes and my batteries died. And you might be able to get that 95 megabytes per second that they specify. You will certainly pay enough for the extra packaging. So I spent lots of dollars. Fortunately, my employer's dollars are not mine. So sometimes I remember I'm an engineer when I'm not doing workplace health and safety inspections. So the first thing you do when you're an engineer is you pull things apart. Here are two SD cards that I pulled apart. The left-hand one was a fake Lenx's chip SD card. It was purporting to be 16 gig. When I pulled it apart to find out why, it's got an 8 gig flash chip on it. Fortunately, they haven't filed off the number. You can't actually see it in the picture. But it's a Samsung standard flash chip. You can get the data sheet off the network and you can see how fast it's meant to go.  The one on the right is a SanDisk. They've layered everything inside layers of PC board and you can't really get at it. So I couldn't pull that one apart, which is annoying. I thought, okay, if I can get the flash chip off, I can desolder that, stick it in a flash reader, and maybe I can get the firmware out and have a look at it. And I thought about that and I thought about that and didn't quite get round to it because it's a bit more much work and my eyes are getting older and I'm not very good at desoldering anymore. But fortunately, in December last year, Bunny Huang did exactly that. And you can get his data there. His code was written for the Novena, so you can plug an SD card of the right sort into a Novena and get the firmware off and do whatever you want with it and even put your own firmware back on. And that's really nice and it opens it up and I'll come back to that later because what I wanted to do in the meantime was make my SD cards faster. To be able to do that, you really need to understand what flash is like inside. This is in all the flash data sheets so I'm going to go through it fairly fast. Flash memory, and I'm only talking about NAND flash now. There's another sort of flash called NOR flash. If you want to know about that, look it up on Wikipedia. NAND flash is divided into pages. Internally, those pages are accessed serially, which means that it looks a lot more like a disk than memory. The typical size of a page is somewhere between 512 bytes and 4K, although I've found some which have a page size up to 128K. Plus some ECC code. You have to have the ECC code there because flash is jolly unreliable. And most of the other things I'm going to be talking about are to try and fix up that unreliability. So your page size is maybe 2K plus some small number of bytes. How many did I say? 112 bytes. That 112 bytes is for ECC. Those pages, which you read and write at once, unfortunately, they're right once. If you want to write a new page, what you need to do is find an empty one, one that's been erased, and write there and mark the old one dead. You can't erase just a page at a time. You have to erase an entire block at a time, which is some number of pages. In this case, this particular one has a 4 meg plus ECC block, but I've seen them up to 8 meg and 16 meg even. And there's a few small ones that have got smaller ones as well. But typically, the class that you can buy off the shelf now have got a 4 meg erase block. What's more, inside that chip packaging, there can be more than one die. Or sometimes on the same die, you can have more than one plane. On a good SD card, you can read from one plane and write to the other without going off chip. And that becomes really nice for garbage collection, which I'll come to later. So this is what it looks like inside. Typical timings, and this is the timings for that Samsung flash chip I showed you just now, to read a, to write a page, that's a, take an erase page and write to it, is 200 microseconds. Typical. To read a page is about 3 microseconds. To transfer it to a form the SD card bus is about 3 nanoseconds per byte. So that's another 12 microseconds on top of the read time. And to erase a block is 3.5 milliseconds. So you really want to do that in the background. You don't want to do it while you're doing all this stuff. So what we do, what they do is inside that flash chip, they put an MCU.  There are about four or five things that MCUs got to do. The first thing is that flash is really, really cheap because they don't waste any silicon. They, when you manufacture silicon, there's almost always defects in it. So if they're manufacturing 32 gig flash chips, say, and 80% of those are bad, they can throw away all the bad blocks and market them as a 2 gig chip. And the MCU there gives you the illusion that it's a 2 gig chip with contiguous blocks in it. The MCU also has to run the state machine between the arrays and the transfers to and from the SD bus. So that's what that MCU has done. It's also got a van buffer because, again, this thing's got bitwise serial access to the various planes and you want to transfer it into bite-wise output. So you need to do some buffering. I don't know how big that van buffer is. One of the manuals, one of the data sheets I read suggested it was about two blocks worth, that's two arrays block size. Another one suggested it was about two pages worth, but they don't actually state it in the data sheets, which is a bit boring. So I've said that flash is unreliable. There are lots of different ways that flash is unreliable. First, you can get single bit read errors. They're fixed by the UCC. Lovely. Secondly, you can get bad blocks. Both ones that you get at the time that the chip is manufactured and ones that develop over time because the chip has a limited life. Typically 100,000 erase cycles before that erase block is dead. The other thing you've got to watch is redisturb. The way that the flash chip works is there's a floating gate in the FET that manages each cell and you transfer charge onto that by tunneling or some other quantum mechanism by raising voltages so that the tunneling is more likely to happen. Now it so happens that when you turn off everything else except for the cell that you want to read, you're also putting voltages on there. They're much lower voltages, but if you do it often enough, sometimes some electrons jump onto that floating gate. So if you're reading from this cell here, the cell next door might get an electron on the gate and start reading a zero instead of a one. That's called redisturb. You can fix that by erasing the block and starting again and you'll pick it up from the ECC being wrong, which means that your MCU you've got somehow to detect that, erase that block, sorry, copy the data doing the ECC correction so that you've got good data, copy it into a new page, erase that block and then start again. So you need another controller. This one is a block diagram from the Silicon Motion controller. The big manufacturers, Sandisk, Toshiba, Samsung and so forth, they manufacture their own controllers and it's totally proprietary and I've got no idea what's in there. Except that I've been told, and this is sort of unofficial, that some of the high-end Samsung ones, the ones that do Wi-Fi and Bluetooth as well, have actually got a high-end ARM chip-running Linux on it. But I've never actually seen one of those cards, so take that with a pinch of salt. What the SD card does is it's got a really, really slow, cheap MCU there, an 8051 typically. They cost maybe 17 cents per SD card. That kind of range. And because it's really, really slow and horrible, you don't want your data going through it, so it's got a completely separate data path down the bottom there. What the data path does is it's got the host controller that actually talks to the bus and that's quite complicated because there are three different modes of talking to the SD card bus. That talks to an SD interface which interprets the commands that you can send over that bus to decide what you're doing with the card. It's got a buffer. In this case, the buffer is 64K plus a bit for the ETC and other out-of-band data. And it's got a flash interface that talks whatever you need to do for the flash card, and there are two standards for flash chips. You just got one and everybody else has got a different one. So most of the third-party flash controllers you can buy will talk either, which is nice. It's also got a hardware ECC generator and check them in there because otherwise you couldn't do it fast enough. So that's the bottom part. Up the top, there's some program ROM. That's actually not ROM. It's usually Nor Flash to let you do executing in place, about 8051 cut things. Not only does it hold the program, it holds some metadata about where on the flash all your housekeeping data is. Because after all, we're moving stuff all around. We've got this great big flash chip there. We might as well use it to hold the mapping tables for where things go and where things are. And we've got a small amount of RAM. That RAM is not big enough to hold all the mapping tables for a large flash chip. So you typically hold those on flash and manage just bits of them in the RAM. So that's what the SD controller card looks like. What it's got to do is present the illusion of having a standard block device there. So it's got to do all the buffering, all the arrays, all the serial to parallel conversion. It's got to cope with a different block size because it presents a 512 byte block interface and the underlying flash could be 2K, 4K, 8K, whatever. It's got to give access to the different planes. It's got to do garbage collection in the background. It's got to do the erasing and the reallocation and the flash translation and the bad block management and blur. Good grief. Lots and lots of stuff. The problem is you go to the back of the SD card package and it's got all these lists of patents and the patent is meant to be an invention disclosure. So theoretically I should be able to go to a patent, read it and find out what it's actually doing. 90% of these ones that I looked up if they've got any documentation at all the US patent has got the claims which is what the people are wanting to protect against and then it says this patent includes in the entirety this other patent which is a Korean one written in Korean not just Korean but Korean patentees but not even my friend who speaks Korean could understand. So I don't know what they do. I'll just talk a little bit about where management because it's kind of interesting. There's two basic ways that you can do it if you're thinking sensibly and some of this has been documented in places like the IEEE conferences. What you want to do is you want to remap blocks when they begin to fail so when you start detecting the ECC going wrong you remap it and the other thing you want to do is spread all the rights across all your raised blocks so that you don't use any one of them excessively and start making it go bad. In practice you do both of those but with one caveat it turns out that for data longevity if you've got something which has been erased once at manufacturer time and write your data on there it will last longer on the flash chip than if you copy it around and erase it all the time. The data seems to last longer so if you want to have something that lasts 10 years you want it on a block that's only been erased one or two times. If you want to just keep it around then yeah. So for data that's going to last for a very long time like the flash program assuming that you've got an SD controller that has the bomb in it then you want to just keep it in one place. The other thing you want to do is in order to avoid that read disturb you count the number of reads and schedule a garbage collection after you've hit some read threshold that read threshold will depend on the chip because I think I've got one more there that's right yeah because manufacturers like ATEM and so forth tend to buy flash on the spot market they get whatever is cheapest and it could be Toshiba one day, Hynix the next Micron the next day after that which means they need to be able to reprogram those controllers so almost all of them have in-system programmability I think people should be fairly scared about that because any time you've got an MCU between you and your data that you don't control the program of it's possibility of a man in the middle attack and we'll come back to that later anyway a fat file system is what you normally get there fat file systems are actually a pretty good match to the standard usage of SD card because if you think about it, you stick it in your camera you fill it up with JPEGs transfer them somewhere else evade the card and start again so you're always writing on a clean card and a fat file system always writes in cluster size units and they're already 64K in these pre-formatted ones and you segment a line from the first partition which means your fat the fat table is in the first evade block and that's pretty nice and there's a conjecture that I have that some flash controls optimize for access to that that area because if you think about what you're doing you're writing to some evade block somewhere and you're writing contiguously and then at the end of the time you need to update the fat table use case will be really nice a fat file system looks something like this you've got a little bit of boot info a boot parameter at the beginning which has got info about where things are then you've got your file allocation table then you've got your root directory and then you've got cluster after cluster after cluster remember that graph because I'll come back to it I just wanted to mention interface speed a little bit you've got three modes of access to a typical card you can either access in a single line mode one bit of the time serial and fit up to about three bit per second or you can flip to a four bit wide system and talk across that or you can do double data rate and do it both on the lower bit sorry the rising edge and the falling edge of the clock which gives you up to 25 megabits per second there's also a standard UHS-1 and UHS-2 and UHS-3 but I haven't actually seen any of those cards which lets you get up to 104 megabytes per second at double data rate that's pretty nice you'll see these class numbers class 2 is 2 megabyte per second class 4 is 4 megabyte per second right class 6 is 6 megabyte per second UHS-1 is usually even higher but there's a caveat the standard says that the class 2, 4 and 6 give you that guaranteed right speed even when the disk is fragmented the flash is fragmented class 10 is only when it's unfragmented which means that for Linux use a class 4 or a class 6 card will usually give you better performance than a class 10 now the controller's got some van buffer the van buffer is smaller than the array size so a guy called Iron Bergman did a whole heap of work to characterize flashes and he discovered that the way that most of these flash controllers work is they keep some number of evades blocks that they're currently writing to and that number varies from 2 up to some fairly large number the size of that buffer seems to increase with the price of the card you can tell by reading something and then reading something adjacent to it after a little while and timing how long that second read takes if it takes a very short time it's probably in the buffer if it takes a long time then it's doing more reading off the flash because of that 12-microsecond delay Iron Bergman wrote a tool, flash bench which you can get off, get it's a really really nice tool all that's in the bin you can just out get and install it and you say flash bench dash A on a directly connected flash controller if you're using a USB adapter forget it this has got to be something that's directly attached otherwise you won't get the timings what it does is it reads a 2K block just before a power of 2 boundary a 2K block that's 1K before and 1K after a power of 2 boundary and a 2K block that's after a power of 2 boundary if you're crossing an arrays boundary then the one that comes before will be relatively cheap the one that comes after will be relatively cheap and the one that goes across is going to be relatively expensive because you've got to close one allocation unit and open another one this is for a Kingston card you'll notice the one of colour in red you've got 300 the diff at the end is the difference between the middle one and the average of the other two and it's 350 odds microseconds until you hit 4 meg boundary where it's 840 so where our conjecture is that the array size for this flash card is 4 meg unfortunately I lost the data from below 32K so for the small ones you can work out what the page size is by a very similar calculation you just look at the one where there's a jump in time and that's really nice because we can characterise the cards that normal file systems have optimised for use on spinning must or they're optimised for ways like XFS and ButterFS they do journals they do snapshots they do transactions but they're all really aiming to reduce the seek time on a single spindle flash doesn't work that way so let's take four SD cards and I've got a guy called Mark Johnson to do this as a summer project last Christmas on 2013 so we took four SD cards and tested them on all the file systems that I had available at that time here with the SD cards I've scaled them because they're all different number of gigabytes by the price per gigabyte the kingston was the cheapest at 80 cents the toshiba class 10x series was $1.20 per gigabyte the sandish extreme $5 and so on and we found the number of open allocation units for the cheaper cards there's two what's more for the kingston class class 10 card one of those allocation units was bound to the file system area so that was always open and then the other one went all over the place the others had nine which was quite nice now I tested these in a sabre light it doesn't have a UHS-1 interface so all the graphs I'm going to show you for these cards have a maximum transfer rate of 24 megabytes per second first thing we did was IO zone and the bottom line is price per gigabyte the line at the side is k per second for an IO zone read test you'll notice that the curve is pretty flat until you get to the kingston one which goes down fast that says that even at 80 cents per gigabyte kingston is not good value write patterns I mentioned before the one on the left is the block trace output for that file system creating a 4 meg file and it goes pretty pretty straight forward the time across the bottom block number across the right there's a glitch at about one second I don't know what that was caused by maybe the process got swapped out and swapped back in again I don't know but you'll notice that it writes pretty linearly and then it writes the fat area on the right hand side EXT4 writes a lot more the scale of this is much more expanded you'll see that the block number on the left goes up to 1.4 times 10 to the 7 instead of being 10,000 it writes the file then it writes the journal then it writes the directory entry and then it writes the superblock with a fairly big delay in between the last two writes for EXT4 the minimum number of writes to create a file is 4 for fat it's 2 enter another file system F2FS this is supposed to be the flash flash friendly file system it's written by Samsung so you'd expect it to work really well on Samsung chips we'll come to that the way it works theoretically it's supposed to use the on chip flash translation layer rather than working against it it's supposed to cooperate with the garbage collection and use those optimizations that were there for the FAT32 so the way it works is it divides the flash into two meg sections you could actually tune that and look for this time so two meg sections and each of those will be written continuously as a log with all of the data in the log before opening the next two meg section it always writes at the log head and those two meg even if you've got a four meg or an eight meg allocation unit would always be aligned with the allocation units and that's nice I'll just go through these, you can read them I'm not going to talk to them, my voice is running out all the metadata is written to the FAT area or the area that would be the FAT area on a FAT file system so if there's an optimization in the control of four FAT this will take advantage of it which is nice the other thing it does is it tries to split the data that's likely to be short term from data that's likely to be long term so that if you've got data that's going to be right once and then sit around forever like a JPEG or something it'll be allocated to something that's then not moved later on it does this by having more than one open two megabyte segment at once it can have up to six depending on the data that's coming in to be written to the place there's also NILFS2 which is an older file system it also is log structured it uses eight meg segments it defers garbage collection to a user process so the user process says what the garbage collection actually you don't say that directly you say what the protection period is for the checkpoints because it continually checkpoints and snapshots the data on the disk again all drivers in the log and there's only one right lockers so let's look at some benchmarks for that one here's postmark postmark reads and writes a lot of small files and as expected FAT32 performs really awful EXT4 performs reasonably well on the Sandish Xtreme Pro and F2FS forms pretty well on all of them the nice thing is that for the Kingston card F2FS performs adequately NILFS performs slightly better but that's a different story again anyway so that's that I've gone through that this year I got an Odroid XU3 it's got a micro SD card so let's try using that instead so I bought a whole heap of more things these are the characteristics I'm running out of time here you'll notice this time the Samsung EVO class 16 gig cards are really cheap at the moment the right hand column is a recommended retail price per gigabyte you can generally buy things a lot cheaper than the recommended retail price so we used Dbench this time you'll notice the Sandish Ultra performs worse than the Samsung Class 6 on every benchmark so on every file system the Kingston, by the way these are ordinal on the bottom in increasing order of price per gigabyte and the Samsung Xtreme Pro performs reasonably well but Nilifest performs really really badly in throughput terms by the way I had a protection period of 10 seconds which is similar to what F2FS provides for these typically you set it up to 2 minutes or something and here we look at the latency curve small is better for these Kingston's are really really slow and Nilifest gives you the best latency of any of the of any of the file systems so in this trade latency for throughput so depending on your workload if you've got a workload that's latency sensitive you might want to use Nilifest if you've got one that's throughput sensitive you might want to use F2FS Postmark write Postmark write performance Nilifest outperforms them that's lots of small files Postmark read throughput also Nilifest is beautiful that I care about like building the kernel this is a make-j8 or no configuration and what we see is that EXC4 apart from that stupid Kingston card performs at least as well as the others and slightly better than most turns out that when you do a make-j8 there's one thing it does at the beginning that's single process it calls git to get the hash of the current head of tree and that actually takes quite a long time it seems that whatever access pattern git does seems to work really well on the EXC4 and really poorly on the other file systems and I haven't managed to investigate that much yet you can also see that if you can spend a reasonable amount and get a Samsung Pro card it doesn't matter much what file system you use it's a equivalent and in fact at that point you're no longer IO bound just CPU bound and that 90 seconds or so is almost the same as if you're building from RAM disk so yeah, that's all my benchmarks there are a couple of other gotchas some cards and files have very high latencies there's another card that I bought that I haven't shown it was a Samsung Evo 24 megabytes per second card it took more than 30 seconds to complete some operations and the MMC driver kicked up its heels in despair and said argh, I can't do anything with this I worked out why it did that a UHS-1 spec says that at 1.8 volts a UHS-1 card can use up to 2.9 watts which means it's drawing just under 2 amps and my poor power supply couldn't cope with that and that's why it was taking more than 30 seconds so watch out for UHS-1 cards and measure their power consumption the other thing is the Nilifest 2 Cleaner can die I've been mudding that Raspberry Pi that I told you about at the beginning for now a year and a half on the Raspberry Pi and every now and then the Nilifest Cleaner D the user space thing that does the garbage collection dies the disk fills up with checkpoint and then everything stops it happens maybe three times in a year and I still haven't worked out why because when the thing hangs you can't get into it to look at any logs and when you pull the card out and look at it it hasn't written any logs because the card's full up so watch out for that one no faster system works well when the flash is full because then you've got to do lots of best fit allocations in what space is left from everything else you're doing F2FS is still fairly immature it hasn't yet been fully optimized for highly concurrent workloads and there is when I run it the benchmarks I showed is from an 8-core machine and when you run it like that you do see some logs being highly contended so there's some work there to do yet it's only been in tree for four or five months the other thing is that any time you've got a process of there man in the middle attacks are possible read the blog entry please I have and don't trust random cars you find in the street I forked the apotec code and ported it to the Saber Lite and to the XU3 for microSD cards so you can grab it and play with it there's also in there a debugger that you can download to the card and do whatever you want poke around, see what's there trace the the original firmware and see what it's doing unfortunately I didn't find any cars that I could actually hack the Samsung, Toshiba and Kingston ones that I had weren't hackable an interesting other fact I pulled those Kingston cards apart because they've obviously used this to me, that's just slow and when you sort of scrape off the epoxy that's there you can read the chip numbers and things and they've actually rebadged Toshiba ones but I reckon there must be second class ones because the Toshiba ones outperform them either that or they put their own firmware on so in conclusion buy a suitable car for your work look out for bargains, you can often get something from APAS auction really really cheaply align your partitions to evade blocks when you grab a standard distribution like Varsbian or the Leonardo distributions for XU3 or whatever they're typically aligned not on the evade block for your card but on a 1 meg or 2 meg boundary realign them and you'll get much better performance and thirdly use a suitable file system NLFS 2 EXT4 or FTFS depending on your card and your workload yeah I've said that and finally have fun try and attack some of these cards they're great and beware of fake cards okay wait for the mic there's one there so basically if what you're saying is that no matter what's printed on the label all cards vary and the performance is all over the place and you're never really sure what you're going to get what is your advice when purchasing cards to get the best performance if you don't know how it's going to perform it depends firstly on how many you're going to be buying if you're buying you know more than a thousand talk to the manufacturer if you're just buying one then find a reputable dealer and deal with them directly as far as you possible I found the cards I buy from Apersorption are reasonably good they are actually what they say they are they're not fakes and again on Apersorption nice apart from the Kingston cards which I found to be uniformly awful actually does more or less represent performance if you've got time and you're going to be deploying these fairly rapidly buy samples try them all out I think the gentleman behind you was next okay right in that case down here I was wondering if you could clarify a little bit the fat optimizations in the system it seems to me that with a fat table you're reading the same page again and again and again doesn't that blow the right cycles right out it would if that's what you were doing but what they do is you've got your your flash file system your fact's there and what it does is it puts that into the van and every time you write it just writes into there and then just before you pull the card out it writes it back how does it know when I'm going to pull the card out oh simple the power pins are longer than the others and it's fast enough that's only a couple of microseconds Murphy says it just isn't well I'm assuming you just lift it and it just disconnects and that's one of the reasons by the way that if you just power off the machine rather than properly unmounting it you're likely to get corrupt cards who's next Madam Chairman where are we next that gentleman how do you know how to realign the two arrays blocks easy first for most cards if you assume a 4 meg you're right so just make the partition on a 4 meg boundary if you want to be absolutely sure use the flash bench tool and measure it also if you get the if you look at the original partitioning on the card when you buy it it will come pre-partitions that will almost certainly be base block aligned before you start so just look at that and use that in all your other partitions a couple of kind of related questions at work we use industrial quality SLC USB sticks which are pretty expensive but so does this USB have the same sort of issues and is there any difference between the MLC in terms of what you've talked about today yes SLC has a longer life than the MLC in general in terms of number of array cycles but apart from that the rest is almost the same just that you've got a different interface at the front you've got the same problem of man in the middle attacks if you look at the bad USB papers from us next year you can see all that stuff yeah so very similar they typically have a higher power budget and more powerful posters just on getting samples and then buying the batch you do want we've brought we've taken samples found a good card brought hundreds of them found half of them worked half didn't packaging and externally identical not the same that's absolutely true if you read Bunny Huang's blog I'm happy that when they were building the Chumbly they bought a whole heap of Kingston ones because they were the cheapest supplier and found that they performed really really badly and they dissolved them in acid found that serial numbers were identical underneath the epoxy and it turned out to be a ghost grey run on the ghost ship run was there another one at the back there somewhere over there your speculation about some of the devices being fat file system are where it's correct more so for USB keys so what they do is they look up from underneath, recognise that fat is being used and change the allocation strategy based on the fact that you're writing fat so you'll get a not necessarily a drop in performance but suboptimal performance if you're not writing fat to one of those devices yeah that's quite probably true yeah any more questions yeah one down here is it sounds like the firmware is potential wait for the mic please so it's for internet land it sounds like the firmware is potentially an issue with some of these devices yes what's the possibility that you can get access to more directly lower layers within the device and like write the driver in the next land yeah that will be a really neat project first you've got to work out how to hack the card or have millions of dollars to talk to the manufacturer and get under NDA what they're doing if you happen to have a card with an apotec controller the hack's already done and that's the one that we I put the URL for out there if you don't have an apotec controller then try it use fuzzing on the programming pins see what you can do when you're talking to the SD card there's a defined standard you send down command bytes and then the data for the command bytes command 63 is allocated to the manufacturer and it's typically that that they use for in-system programming so try fuzzing it see what you can do oh yeah and go to Shenzhen and buy dozens of them anymore okay I think we're done are we having fun yet? thank you very much for the talk and it's a small present