Hello, everyone. Graph.no. My name is Rui Huang from MD Linux team and focus on the multi-components in the Linux kernel, including the CPU power measurement and GPU DIM graphics compute support over about 10 years. This is my first time to present talk in the LinuxConf at OpenSource Summit. So it's my great honor to have the opportunity to talk about the new CPU frequency control mechanism on Linux here to all of you. In fact, it is a long time since we prepared the solution last year. Then we started to upstream the kernel module at September and take 7 rounds of public upstream code review and then until about earlier this year, the initial patch set was finally accepted by the kernel 5.17. And today, I would like to go through for technical sharing with the overall design and implementation of this solution. Thank you so much to be here and listen to my introduction and feel free to contact me if you have any questions. Okay, move to the background. This function is required from the Chrome project with the season shapes. That is the first MSR solution support for the collaborative processor performance control. Let's say CPPC, because the Chromebook would like to have a better performance per broad scale in their new product. And at the same time, why we were working with wolves of engineers to enable StingDec on the AMD platform last year. We were tuning the Sting games performance on the Vulkan D3D Proton. They found two issues on the AMD processors. One is something wrong with the CPU frequency prints and the other is slow motion on the horizon zero jaw. We spent a long time to investigate them and found quite a few problems for ACPI CPU free driver on the AMD processors. So we thought the ACPI CPU free driver might not be very performance and power efficiency on the modern CPU platforms. Okay, let's talk about existing the kernel frequency control here. As we know, Linux kernel provides a CPU free framework to use the kernel terminals on-demand and the schedule to use such as this kind of policy to control CPU clocks. And many years ago, Intel implemented an ACPI based CPU free driver. And it is absolutely a very good driver to provide a general solution on the Lexi Intel processor before the Sandy Bridge. Then they switched to an Intel-specific PSD driver in recent CPU series. However, current AMD CPU platforms are still using the ACPI based CPU free driver to manage the CPU frequency and clocks with switch only in three PSD. The ACPI based CPU frequency driver develops based on the Intel platforms and let's bring some potential issues on the AMD processors. The AMD hardware also provides a former-based CPU clocks dynamic power measurement. It is also to control the processor frequency. We call it DPM. The ACPI CPU free driver is not avail of the AMD CPU clocks DPM in the former. So they would have the conflict to impact the final target frequency. So we plan to design and implement a new AMD CPU free control proposal. This proposal is expected to handle the hardware functionalities such as the AMD CPU SMU former and ACPI CPPH. The AMD processor provides the MSR register as the back-end mailbox for the frequency control. Because MSR is a low-latency register model that is faster named ACPI AML code interpreter. Above all, we decide to implement a new MDP state driver instead of ACPI CPU free driver for AMD platform. It is able to use a finer gram CPPC frequency range instead of traditional ACPI 3P states to control the CPU frequency. We can use even leverage the kernel governor such as the schedule tools to predict the workload to calculate the reasonable design performance values with the Linux CPU CFS scheduler. We mainly use the schedule tool governor to optimize the solution and manage the hints to the SMU CPU clocks DPM to calculate final target frequency. This page expands the many methods of the frequency control in the former. Let's formula is got from the SMU former and it is the algorithm less the SMU former to adjust the frequency at low level. I probably cannot go details of the algorithm because we can know the target frequency is mainly decided by the activity here. The activity in fact indicates the maximum C0 resistance during the DPM cycle. That means the input is decided by the CPU idle driver in the Linux kernel. It's very similar with on-demand governor in the Linux kernel and I will expand later in this discussion. These four performance values are defined for MDP state performance capability entries. They are the performance scales which is mapped as different processor frequencies. The different processors type have different frequencies. MDP state solution introduces continuously performance scaling between the highest perf and the lowest perf. The highest perf is similar with the both state before and the nominal perf is similar with P0 states. And the processor frequency can be dynamic adjusted during this range. This is another four performance values are defined for the MDP state performance control entries. The performance scales which are mapped at different processor frequencies as well. Users can assign the performance range as the hints to the hardware. The hardware can control the real-time frequency according to the hints that MDP state driver provides. And the fourth scale is different. It is provided hints if the driver wants to bounce towards the performance or energy efficiency. We are still working on this feature at this moment and continue upstream the port once it gets wirefiled. This kernel has several governance such as on-demand performance, power-safe user space, and the schedule to control the general frequency change for most CPU processors. Performance and power-safe set highest and lowest performance goals statically for processor cores support. The on-demand governance is mainly used for most of the current processors besides the modern MD Zen Serial CPU. This governance sets the CPU frequency depending on the current system workloading. Schedule 2 governor expires the better integration with the Linux kernel scheduler. And the load estimation is achieved through the Schedules for Entity Loading Checking method, which is also provides the information about recently workloading. The MD-CPU Frick design is to create a new kernel module, named MDP state driver, under the CPU Frick subsystem. This module can access the CPPC-related MSR registers, which are provided on some of the Zen 3 and later MD processors, which includes the CPPC functionality. The SMU former can detect the hints less MD-PC state passes while the MSR registers, and responds the request into the hardware to control CPU clocks according to the performance goals from the MD-PC state module. The CPU cores estimate CPU usage statics over the last period, which is similar with the SMU former method to monitor the C0 resistance. Then decide frequency is increased or decreased. However, these kinds of CPU statics are mainly controlled by the ACPI processor idle driver in the Linux kernel. Let's use the MMWate FFS style translation to set CPU from C0 to C1. Current Linux kernel is mainly using the completed fail scheduler. We call it CFS to manage the processor scheduling. Because the scheduler is aware of all the processors, including the kernel and the user space inclusion status, the CFS scheduler provides per entity loading checking method. We call it PLEPELT to manage the real workload at real time. The schedule too works based on the DVFS for task managers by the CFS. The DVFS is dynamic voltage and frequency scaling. The real time and the deadline scheduler tasks are always running on the highest frequency. This is user mode CPU power extension tool to manage the CPU frequency subsystem running status. We also asked the MDP state and ACPI CPC lib support in this tool. After that, we also introduced the MDP state tracer in the Linux kernel as well. It is very useful to monitor processor running status that can be used for the performance and power tuning at platforms. An MDP state driver is brand new and can also work on some of all the same processors. We introduced the unit test suite into the self-test as well. That is to make sure to have good functionality on current system BIOS and former supported platforms. As we know, the MDP processors in the CPU MDP processors are controlled by the SMU former both on the CPU and GPU portion. Each MDP CPU and GPU are all used SMU DPM. We call it dynamic power management function to control the CPU, graphics, data fabric, clocks. The arbiter in the SMU former is the state machine to manage the clocks control. Please take a look at the two pictures which are the program sequence, that's how to get the CPPC capabilities data of the highest, nominal, lowest-long-linear and lowest-performance and expose them to the ACPI interface or MSR registers. Another is the program sequence, that's how to respond to the request hints of CPPC such as the max, manning, design performance and EPP values. These are the dependency components that we need to take care of for the MDP state driver. Firstly, the first is the CPU idle driver. The MDP CPU is using the M1 weight FF style translation to set CPU from C0 to C1. Actually, the CPU idle framework is managed C0 resistance percentage in the system. We provide a slide that I just mentioned. The resistance indicates the activity here in the SMU former. That is the key impact factor of the final target frequency. Actually, the SMU former-based frequency control also depends on our OS kernels, the CPU idle driver. Another component are the kernel terminals in the CPU free driver. As suggested by the kernel maintainers, we are trying to optimize the Schedule 2 and on-demand terminals for the MDP state driver. For example, Schedule 2 can predict the workload directly, which is more efficient in seriesistic. But we still do more tuning in the real hardware. We still need to tune the new CPU free driver on our existing MDP platforms. The Linux CPU free core offers a standardized interface for the CPU free subsystem. We use these interfaces to implement a new kernel CPU free driver, which is named MDP state. The MDP state model is to instead of the Lexi ACPI CPU free module implement CPPC function in the MDP platforms. And it is to manage CPU frequency and performance with the MDP state ATIs with multiple Linux kernel terminals. We are using the two different processor identifiers to know whether the processor supports the MDP state module or not. One is ACPI CPC object in the ACPI table from system files. And another is CPC CPU ID. The CPU ID is used for checking the MSR support on the current processor. The MDP state driver implements CPU data structure to store the private MDP specific information and the function callbacks. And we implement the MDP state register write and read helpers for the capability and the request interfaces. And this driver implements a free QoS request instance for the free constraints. The constraint structure sets the limit range between the maximal and the minimal frequency, which register into the Linux PM QoS framework. The QoS is quality of service. And then we also implement the CPU performance scaling driver RST restructuring the texture kernel documentation, which is to introduce this driver details and please feel free to access this link to know the details. We will maintain this documentation as well. Let's continue to look at MDP state driver. This module is MDP state driver instance to implement the general operations in the callbacks functions. It can use a relative performance and scaling control that is dedicated for the MDP state interfaces on the MDP processors, such as the target and adjust the frequency update functions. The target is the traditional callbacks, which is used for the on-demand and schedule UTO governor. And the adjust perf is the fast-switch function that is only for the schedule UTO governor. In fact, they are all used for mapping the functions in the governors. Then the kernel governors can manage the P-State performance hints with the MDP state module for each processor core. In future, we plan to implement kernel parameters, how to switch the different frequency management policies and customize the different management policies for specific products. We will also implement the UPP, energy performance preferences, pre-reference control and pre-for-core and fast CPPC. I also would like to discuss with the Linux Power Management developers for the further support in the Linux Promo conference as well. There are two static trace events that can be used for the MDP state diagnostics. One of them is the CPU frequency trace event, which is generally used by the CPU freak. We can use these traces to monitor the real-time CPU frequency of each logic core. And another is MDP state perf trace event, that's specific for the MDP state driver. It can be monitored, all the performance goals, which MDP state programs to the hardware side at the wrong time. This is an example of the trace event, that's MDP state module, that's definitely very useful debugging information for performance tuning. Then we can leverage the trace event in the Linux kernel to develop a user spacer, TracerTour. The TracerTour is a Python script, and can record and parse the Tracer logs. This feature is to check the latency between why we have a workload and why the CPU frequency is increased. You can look at this picture, why we have a workload, then the frequency is increased. Once the application is gone, the workload is also gone, then the frequency goes down for the very low level at this moment. We can monitor the wrong time data, every time we write into the performance goal, into the hardware registers. At the same time, we can also check the wrong time CPU frequencies. At the same time, please see this kind of data from Excel file. This is the link of this tour for the MDP state Tracer detail implementation. MDP state exposed several global attributes files in the CFS to control its functionalities at system level. These attributes file located under the CPU free subsystem. This is the part of the CFS interface. They indicate the highest and nominal and lowest long-lanion and lowest processor performance values and frequencies. These APIs are used for the CPU power tour. In MDP state driver, we only implement the different frequency and performance, which is not included in the CPC live. This is the CPU power tour and the library. In this picture, we introduce the CPU power tour and the TurboState support. The CPU power tour and the library use the space tour to query and assess the CPU free APIs. This tour is suggested by the Linux community and widely used by the Linux world as the processor power configuration applications. It adds two components, let the MDP state performance and the frequency level helpers and MDP state CFS operations handling. We also add general ACPI CPC CFS APIs into this library. The user can control the CPU power tour to manage the MDP state module. The TurboState will be the next step we would like to support for displaying the frequency and power consumption, idle status and so on. That's definitely a very famous tour in the community. We will introduce unit test support. The UT is the unit test test module to test the MDP state driver functionalities. It is for verifying the support of the system BIOS and the firmware on existing CPUs. You know the MDP state driver is new, but it can run on many old MDP CPU platforms. So we would like to have a unified test baseline that can be running on all the CPU platforms that support it on the CPC. And we will add more CPU benchmark automation testing, such as T-bench and GitSource in the test module. It's leveraged kernel self-test framework to implement it. Then each developer and user can easily test performance data in their platforms. And also easy to test all the existing CPU platforms with CPC functionalities. So it's definitely a very useful tour. And we will add more test suites and test steps into the kernel repo. We are using the RAPL, the running average power limit interface to calculate energies of current CPU patch power. And we are using the perf tour to monitor the energy consumption while we are running the any CPU benchmarks. The APIs is already implemented in the latest kernel right now for all the ZEM-based processors. This approach can be used for the performance per-rat testing on the CPU platforms. And this is the guidelines of the formula that we calculate performance per-rat scaling for each benchmark test. Community provides a paper to explain how could we test and measure the performance per-rat data. And I didn't want to go into very details here if you have any interest, we can discuss it offline. Okay, we pick up the MD-CZEM processors to do the sound performance per-rat testing on the multiple CPU benchmarks. This is a T-bench test result. They are the average data. Let MDP state got 3% drop in our Vista schedule tools and 2% improvement with the performance governance. And these are the 15 times average data and pictures here. And this is the test environment we get from this benchmark. And we were also provided the T-bench test suite for the MDP state UT. And then the user can verify it in their platforms as well at the same time. You know, the MDP state is still brand new even we have a better design, but we still do more power and performance optimization in the existing BIOS firmware and kernel driver. And that is the GitSource benchmark test. They are the average data here. The MD-P state can got very positive data in this kind of benchmarks. And the MDP state can got about 24% improvement with scheduled UT and 30% with on-demand governance. So we can have a good data on this kind of benchmarks. But we still need to have more issues to investigate how can we get the performance data in this kind of benchmark. Not only save power, performance, perhaps we need to use a balance during the performance and power side. Okay, this is the speedometer benchmark which is required from the Chromebook. And the schedule too, governance performance for VAT is almost the same. But we still need testing and optimizing the power base. Sorry, we have 9% improvement in on-demand. But they faced 3% job in the performance governance. So the Chrome is default using on-demand for the MD platforms. So we can have a bit improvement in the Chromebooks here. But we still need to do more optimizations. The MD-P state is still new and we still continue testing and optimizing the power base on the Chrome software platform. Okay, finally, let's talk about Karen's status of MD-P state driver. Actually in design, both on hardware and software design, the MD-P state CPPC provides a final-grammed performance range instead of the Lexi ACPI-P state which leveraged for the ACPI-3 driver. Less the manning improvement from this driver. But so far, we still have a long way to go. We need to do more testing because it is new. We can support this kind of driver in the existing platforms, Zen2 and Zen3 platforms as well. And we also spend a long time to go upstream. In the length of kernel 5.15, we have the MD-P state driver support. And in kernel 5.18, we have a CPU power tool and support and the MD-P state tracer tool support. I think following the kernel 6.1, we were introduced the new MD-P state unit test support. And in the next, we were introduced the EPP and the preferred core support in the future. We can see some walking staters from Mr. Bureau Walking Ripple, Git Trace. And you know, at last, it is a brand new driver, MD-P for MTP state. So we still need to have a long way to go. Currently, we face some challenges. We still have some challenges. The first one is we need to resolve the performance and power issues on the multiple benchmarks. You know, we still face the share memories issue, performance, power drops, which compare with the ACPI CPU flick driver. In these kind of processors, we definitely still have some issues. And the second, we still need to cover all the existing CPU platforms with different types of processors. As the mobile, desktop, server, and so on. And the third, is we still need to optimize the power on the steam lag for gaming, that is the proton. Because, you know, the protons were leveraged while, the while were talked to the CPU flick APIs directly. So maybe we can do some good transactions between the proton and the CPU kernel, CPU flick driver for the MD platforms to make the, we have a better gaming power and performance in future. And that's all for, actually less the challenges so far. We will try to adjust this kind of concern for this kind of challenges for us. So, and yeah, that's all for my side. These are the performance links for the whole pictures, MDP state driver design. Feel free to ask me any questions and then we can discuss here or mail me is welcome. Any questions are welcome for me. And also, thank you very much for listening. And that's all for my side.


Everybody force and Jake here up in 22 23 select basketball hobby and H2 Six-box mixer number four random teams. So it's three of each year selects hobby and H2 from 22 23 Good luck round of one now on one and one I think mine won So let's go ahead and random then we'll rip Jake was talking to you guys while the intro was going I didn't tell him I started to break All right. Good luck on the random Go seven times Five and two how you doing Noah by the J spots. Yeah, Jay I mean if you're collecting the J's not so bad because they're cheap, but cheap for a reason It didn't get both ended to get both a sign name seven times one two three four five six and seven Zack s down to Eddie and teams now one Two three four five six and seven The pool dude Celtics down to Blazers Jacks it's NFL fans. What do you what do you want? This is how NFL fans are Irrational Zack s Celtics Rick s Nick's Thomas M Hornets Nick M. Bucks Dan F calves Zach s jazz Nick M Mavericks Thomas C wizards Michael B. Sons Alex W Wolves or you'll see pistons Derek K Warriors James F Hawks Alex W Spurs Tomiyuki Nuggets sh Lakers Thomas C Pacers Joseph B Raptors Jonathan a thunder Michael B Grizzlies Nam Q Kings Scott F heat oh yet with the Rockets Zack s Sixers David F Pelicans William C Bulls Luis s magic Jonathan a Clippers Brandon G Nets and Eddie L Blazers What should I just start this break intro as he's talking Dude, I had a whole thing going before it's because I started intro Really Yeah, Jay's been trading their prospects you looked at me like are you having serious right now? You're just like he let it go to I Let him say I thought he was gonna say the one thing that he kept going. Oh, hey, well This is this happening. All right. Good luck links and chat finals anytime late sports cards calm one to go one to go one to go one and one Need to go on flawless 90 go in the flawless mixer is Yeah, we do have pickle Rick back there Lane. We're tired to acquire pickle Rick Down to seven guys. I got some good prospects. They just end up trading a lot of them. They should probably keep some Let's go concourse MPJ nugs Trader Marino Light blue premier level rookie is Christian Brown from Denver 55 of 99 silver concourse is KD Sons Who did a Brown post drive somebody last night? He flew for that dog shade on sharp Premier level white is gonna be De'Aron Fox Kings number 40 of 75 Light blue LeBron James courtside 89 of 99 Lakers Silver concourse Scotty Barnes Raptors White premier level rookie Christian Brown for Denver. There you go 38 to 75 another one for Denver Light blue of Lillard out of 99 for the Blazers Hort side silver Scotty again. I don't know for soda. That might not be worse Chet Holmgren rookie concourse Being being able to trade but know it is all-time low value white rookie Christie Lakers 4 of 75 Red Yanis courtside bucks out of 49 Silver concourse Caleb Houston rookie magic Hey, it's already said Blue jays is gonna be lost in the pack this year I don't think I think they're probably the third best team that division They could overpay To make a reaction to Randy John he could be the best pitcher ever concourse just go Devon Vasell for the spurs that is high praise Benedict Matheran Murray got bread in the case you show hey, right RJ Barrett and next to 49 Silver courtside rookie Christian Brown for Denver. No, it's a lot of Christian Brown here. We got a patch auto in H2. Here we go It's King and Murray Kings rookie autograph or not not again. I thought they're only RPA's in this All right, King's rookie auto King and Murray Concourse blue is gonna be Kawhi Leonard Clippers 27 of 99 Concourse white as Jalen Suggs magic Number 30 of 75 and a silver concourse rookie is Mark Williams for the Hornets All right three hobbies now try color concourse of Van Vleet for the Raptors a Silver concourse rookie is Roddy for the Grizzlies. That's just crazy high praise dude. Hopefully is that good for y'all? tie-tie Washington to 99 rookie Jersey card for the Rockets silver premier level to soon move Thank you. Thank you in their purple Tyrese Halliburton Pacers premier level Number 63 of 99 scope concourse Celtics Marcus smart Got tie-tie Palo concourse tri-color concourse garland for the Cavs Chet Holmgren rookie court side in Ivy gold old snapshots Nicola Yokech The nugs seven to ten sick Yokech Grads Denver killing it here one and one is sold out Go seven and nine on the last two breaks guys Yes, let's hope all the flawless mixer. What's up Higley tops now Star Wars. Yeah You're gonna get a lot of a Star Wars cards in tops now. All right shade on Light blue concourse is marketing jazz number 12 out of 299 Loco It's like future it was my jank thunder concourse over Desmond Bain for the Grizzlies Demption got silver rookie court side AJ Griffin Hawks and SGA X factor memorabilia signatures gold big card thunder Card thunder gold jersey auto SGA, I'm sure that'll have a nice patch on it, too If it's gold, hopefully will orange premier level Marcus smart Celtics. It's number 56 of 65 59 65 Josh Hart to 99 on the autograph of the Knicks in flight auto Premier level tri-color it was Monjang thunder. Do you want to make sure that that is a? Thunder SGA. What is it X factor and really signature card number two? Yeah, it is thunder. Yep premier level scope is hardened for the 76ers Is that a jagged turnover that time during Palo concourse silver concourse John Moran Grizzlies. He's back soon Anthony Simons to 75 Jersey card Portland Silver Malcolm Brogdon concourse red rookie Yovitch for Miami 77 of 199 Jade. There's a tri-color market and jazz concourse. I hope we can put it all together in the bigs Tri-color premier level Zion for the pelicans Premier level blue Jalen Green for the Rockets number 32 of 249 Let blue concourse Dyson Daniels rookie for the pelicans there to 299 curry Silver concourse Jalen Terry for the Bulls, maybe we'll pull Steph Curry Tom's on this auto one of one next week. Come on Dude Keegan married at 249 another one for the Kings congrats Kings rookie out of there. He went what was it last night Keegan went off Silver Jalen dirt rookie premier level pistons. How did he fumble that? That was crazy Concourse blue is Yannis for the Bucks number to 249 Kendall Brown Jersey auto rookie to 99 Pacers Tri-color sex dinner for the jazz One more box 30 people in this break right here. Let's see what the punchline is shady Steven is take the landing concourse neon of MPJ for the Nuggets number 42 of 75 Bumble gum Fingers, but fingers going this one was 69 99 per spot Try color premier level hero for the heat White concourse of a cell for the Spurs 144 of 149 Baby is back seven a good game not tonight. Higley not Lamello to 99 for the Hornets Jersey card Stop Robert. Are we doing tonight? Hey, he's back Try color of a cell Spurs load them up in fire, baby. It's a game war on the mellow by the way Dude Baker tore him up today Blue is RJ Barrett to 249 Nick's concourse Try color Ingram concourse for the Pelicans scope concourse to soon move Bulls Shade on on the court side Keegan rookie Nice Dalen Terry on the tie-dye rookie premiere level Chicago 225 It was my jang silver concourse Desmond Bay and Grizzlies Silver concourse rookie for the Pistons Jalen Duren No, no, I think don't say that dude Come on man Stop How low rookie Jersey to 75 Orlando you went off today, dude No one else for the magic showed up, but he went off ice-card for Orlando rookie premier level silver as Baldwin for the Warriors Never have Papa John's again Who's the scrub team we're playing next so we get a win against premier level silver Jimmy Butler heat And redemption and it's gonna be Robert Williams the third X factor memorabilia signatures. I'm gonna check that one Pretty sure that Celtics though He was only with the Celtics before this year, right? Was he X factor No, I don't be a blazer in this no Celtics wanted to make sure he was only on he's only been on the Celtics and he traded a blazers, right? Celtics on that one quick recap Had relics of Lamello tie tie Simons Paolo Redemption for the Celtics was Robert Williams the third X factor memorabilia signatures Auto Keegan Murray another Keegan Murray that one's number to 249 the H2 one was not numbered Josh Hart and next the 99 Jersey auto 299 is Kindle Brown with the Pacers Thunder redemption gold X factor Sanity remember Billy a SGA big-time redemption there And a gold Yokech on the snapshots for the nuggets that one's seven of ten Thanks again everybody on a two one and one


Welcome to the Nutrition Facts Podcast. I'm your host, Dr. Michael Greger. The coronavirus pandemic has made many of us very aware of the importance of maintaining and improving our health. Make that your silver lining. Because the more positive change we can make to our diet and lifestyle, the better. Today, it's part three in our four-part series on Strokes and Diet. In our first story, we investigate the drop in stroke risk as the Japanese diet became westernized. Stroke was the leading cause of death in Japan, but the mortality rate sharply decreased as they moved away from their traditional diets and started eating more like those in the West. So maybe there was a protective effect of all the extra meat and dairy they started to eat.  At the same time, their stroke rates were going down. Protection from stroke by eating animal foods? Surely not, commented a noted Loma Linda cardiology professor. Many vegetarians, like myself, have almost come to expect the data to indicate that they have an advantage, whatever the disease that's being considered. Thus, it is disquieting to find evidence in a quite different direction for at least one subtype of stroke. Can dietary saturated fat like that found in meat and dairy be beneficial in the prevention of stroke risk? There appeared to be a protective association, but only in East Asian populations. High dietary saturated fat was found associated with a lower risk of stroke in Japanese, but not in non-Japanese. So what was it about the traditional Japanese diet where the westernization of their diets made things better when it came to stroke risk? Well, at the same time their meat and dairy was going up, their salt intake was going down. The traditional Japanese diet was packed with salt. In fact, they had some of the highest salt intakes in the world, like a dozen spoonfuls of salt a day. Before the widespread availability of refrigeration, they all sorts of salted, pickled, fermented foods from soy sauce to salted fish. And in the areas that had twice the salt intake, they had twice the stroke mortality. But when the salt intake dropped, so did the stroke death rates, because when the salt came down, their blood pressures came down too. And high blood pressure is perhaps the single most important modifiable risk factor for stroke. So there's no big mystery why the westernization of the Japanese diet led to a drop in stroke risk. When they abandoned their more traditional diets, their obesity rates went up. Their diabetes and coronary artery disease went up. But as they gave up the insanely high salt intake, their insanely high stroke rates correspondingly fell.  Their stomach rates came down beautifully as they westernized their diets away from salt-preserved foods. But of course, as they started eating more animal foods like dairy, their rates of fatal prostate cancer, for example, shot through the roof. Compared to Japan, not only does the US have 7 times more deaths from prostate cancer, but 5 times more deadly breast cancer, 3 times more colon cancer, and lymphoma mortality, and 6 to 12 times the death rate from heart disease. Yes, Japanese stroke and stomach cancer rates were higher, but they were also eating up to 6 spoonfuls of salt a day. That would seem to be the most likely explanation rather than some protective role of animal fat, as eventually acknowledged by the official Japanese guidelines for the prevention of cardiovascular disease. Now, one of the Harvard cohorts found a protective association between both saturated fat and trans fat for hemorrhagic strokes, prompting a sigh of relief throughout the cattle-producing Midwest, even though the researchers clearly concluded that, of course, we all have to cut down on animal fat and trans fat for the heart disease benefit regardless, but looking at another major Harvard cohort, they found no such protective association for any kind of stroke and put all the studies together and zero protection across the board. Observational studies have found that higher LDL cholesterol seems to be associated with lower risk of hemorrhagic stroke, raising the possibility that cholesterol may be a double-edged sword, decreasing the risk of ischemic stroke, but increasing the risk of hemorrhagic stroke. But low cholesterol levels in the aged may just be a surrogate for nutrient deficiencies or a sign of debilitating diseases, or maybe they're on a combo of cholesterol-lowering drugs and blood thinners, and that's why we tend to see more brain bleeds in those with low cholesterol. You don't know until you put it to the test. Put together about two dozen randomized controlled trials and the lower your cholesterol, the better when it comes to overall stroke risk. With no significant increases in the risk of hemorrhagic stroke risk, with lower achieved LDL cholesterol levels, the genetic data appears mixed, with some suggesting a lifetime of elevated LDL would give you a higher hemorrhagic stroke risk, and other data suggesting more of a double-edged sword effect. But, you know, any possible excess of bleeding stroke with lower cholesterol is greatly outweighed by the protective effect against the much more common clotting stroke, not to mention heart disease. Perhaps on the order of 18 fewer clotting strokes for every one extra bleeding stroke with cholesterol-lowering. If there's some kind of protective factor in animal foods, it is to be hoped that a, you know, diet can be found that still protects against killer number one, heart disease, without increasing the risk of killer number five, stroke. But first, we have to figure out what that factor is, a hunt that we'll continue next. In 2014, a study on dietary protein intake and stroke risk found that greater intake was associated with lower stroke risk, and furthermore, it was the animal protein that appeared to be particularly protective. Might that help explain why vegetarians were recently found to have a higher stroke rate than meat-eaters? Animal protein consumption increases the level of a cancer-promoting growth hormone in the body, known as IGF-1, Instantly Growth Factor 1, which can accelerate the progression of precancerous changes to invasive cancer. High blood concentrations are associated with increased risk of breast, prostate, colorectal, and lung cancers, potentially explaining the association between dairy milk and prostate cancer risk, for example. But there are also IGF-1 receptors on blood vessels, so maybe IGF-1 promotes cancer, but also brain artery integrity? People who have strokes appear to have lower blood levels of IGF-1, but it could just be a consequence of the stroke rather than the cause. There weren't any prospective studies over time until 2017, and indeed, higher IGF-1 levels were linked to lower risk of stroke, but is it cause and effect? In mice, the answer appears to be yes, and in a Petri dish, IGF-1 appears to boost production of elastin, a stretchy protein that helps keep our arteries elastic. Higher levels are associated with less artery stiffness, but acromegaly patients like Andre the Giant, those with excessive levels of growth hormones like IGF-1, do not appear to have lower stroke rates. In a more recent study of dietary protein intake and risk of stroke, looking at a dozen studies of more than a half million people compared to only seven studies, with a quarter million in the previous analysis, found no association between dietary protein intake and the risk of stroke. In fact, if anything dietary plant protein intake may actually decrease the risk of stroke. However, those with high blood pressure who have low IGF-1 levels do appear to be at increased risk of developing atherosclerosis, thickening of the artery walls leading up to the brain, but no such association was found in people with normal blood pressure. So there may be a cautionary lesson for vegans here. Yes, a whole food plant based diet. To down-regulate IGF-1 activity may slow the human aging process, not to mention reduce the risk of some most common cancers that plague the Western world. But perhaps the take-home message should be that people who undertake to down-regulate IGF-1 activity by cutting down on animal protein as a pro-longivity measure should take particular care to control their blood pressure, to preserve their cerebrovascular health, the health of the arteries in their brain. In particular, they should keep salt intake relatively low while ensuring an ample intake of potassium to keep their blood pressures down. So that means avoiding processed foods and avoiding adding salt, and in terms of potassium rich foods, beans, sweet potatoes, and dark green leafy vegetables. So might this explain the higher stroke risk found among vegetarians? No. Why? Because dairy and egg whites are animal proteins, to only vegans have lower IGF-1 levels in both men and women. So low IGF-1 levels can't explain why higher rates of stroke were found in vegetarians. So what is it already? I think the best explanation for the mystery is something called homocysteine, which I'll cover next. Leonardo da Vinci had a stroke. Might his vegetarian diet been to blame? His stroke may have been related to an increase in homocysteine level because of the long duration of his vegetarian diet. A suboptimal intake of vitamin B12, which is common in those eating plant-based diets, unless they take a B12 supplement or eat B12 fortified foods regularly, can lead to an increased level of homocysteine in the blood, which is accepted as an important risk factor for stroke. Excepted may be overstating it. There's still controversy surrounding the connection between homocysteine and stroke risk, but those with higher homocysteine levels do seem to have more atherosclerosis in the crotted arteries that lead up to the brain, compared to those with single-digit homocysteine levels, and do seem to be at higher risk of clotting strokes in observational studies, and more recently bleeding strokes, as well as increased risk of dying from cardiovascular disease, as well as all causes put together. Even more convincing is the genetic data. About 10% of the population has a gene that increases homocysteine levels at about two points, and they do appear to have significantly higher odds of having a stroke. Most convincing would be randomized, double-blind, placebo-controlled trials to prove homocysteine lowering with B vitamins can lower strokes, and that indeed appears to be the case for clotting strokes, strokes with homocysteine lowering interventions more than five times as likely to reduce stroke compared to placebo. Ironically, one of the arguments against the role of homocysteine in strokes is that, assuming that vegetarians have lower vitamin B12 levels, then the incidence of stroke should be increased among vegetarians, but supposedly this wasn't the case, but it had never been tested, never been studied until now. And vegetarians do appear to be at higher risk, and no wonder. It's about a quarter of vegetarians and nearly three-quarters of the vegans studied were B12 depleted or deficient, resulting in extraordinarily high homocysteine levels. Why so much B12 deficiency? Because only a small minority were taking a dedicated B12 supplement, and unlike the U.S., B12 fortification of organic foods isn't allowed, so while U.S. soy milk and other products may be fortified with B12, U.K. products may not. We don't see the same problem among U.S. vegans in the Adventist study, presumably because of the B12 fortification of commonly eaten foods. It may be no coincidence that the only study I was able to find that showed a significantly lower stroke mortality risk among vegetarians was an Adventist study. But start eating strictly plant-based without B12 fortified foods or supplements, and B12 deficiency can develop, but that was only for those not eating sufficient foods fortified with B12. Those eating plant-based not being careful about getting a regular, reliable source of B12 have lower B12 levels and, consequently, higher homocysteine levels. The only way to prove vitamin B12 deficiency is a risk factor for cardiovascular disease and vegetarians is to put it to the test. And when researchers measured the amount of atherosclerosis in the crotted arteries, the main arteries supplying the brain between vegetarians and non-vegetarians, no significant difference was found. They both looked just as bad, even though vegetarians tend to have better risk factors such as cholesterol and blood pressure. They should just be 12 deficiencies playing a role, but how do they know? Some measures of artery function weren't any better either. Again, they surmise vitamin B12 deficiency overwhelming the natural plant-based benefits. Yes, the beneficial effects of vegetarian diets on cholesterol and blood sugars need to be advocated, but the necessity to correct vitamin B12 deficiency in vegetarian diets cannot be overstated. Sometimes vegetarians did even worse. Worse artery wall thickness, worse artery wall function, raising concern more than a decade before the new stroke study about the vascular health of vegetarians. Yes, their B12 was low. Yes, their HOMOSysteine was high, suggesting that vitamin B12 deficiency in vegetarians might have adverse effects on their vascular health, but what you would need is an interventional study, where you give them B12 and see if that fixes it. And here we go. Double-blind, placebo-controlled, randomized crossover study, and the title gives it away. Vitamin B12 supplementation improves arterial function of vegetarians with subnormal vitamin B12 status. So that may explain it. Compromised vitamin B12 status among those eating more plant-based due to not taking B12 supplements or regularly eating vitamin B12 fortified foods may explain the higher stroke risk found among vegetarians. Unfortunately, many vegetarians resist taking vitamin B12 supplements due to misconceptions like holding on to the old myth that deficiency of this vitamin is rare among plant-based eaters. A common mistake is to think that the presence of dairy products and eggs in the diet as in a lacto-ovo-vegetarian diet can still ensure proper intake of B12 despite excluding meat. And now that we may have nailed the cause, maybe future studies should focus on identifying ways of convincing vegetarians to routinely take vitamin B12 supplements in order to prevent a deficiency. We would love it if you could share with us your stories about reinventing your health through evidence-based nutrition. Go to nutritionfacts.org slash testimonials. We may share it on our social media to help inspire others. To see any graphs, charts, graphics, images, or studies mentioned here, please go to the nutrition facts podcast landing page. There you'll find all the detailed information you need, plus links to all the sources we cite for each of these topics. For a vital timely text on the pathogens that cause pandemics, you can order the e-book, audio book, or now hard copy of my latest book, How to Survive a Pandemic. For recipes, pre-order my How Not to Diet Cookbook out this December. It's beautifully designed with more than 100 recipes for delicious and nutritious meals, and all proceeds I receive from the sales of my books go to charity. NutritionFacts.org is a non-profit, science-based public service where you can sign up for free daily updates and the latest in nutrition research via bite-sized videos and articles. Everything on the website is free. There's no ads, no corporate sponsorship. It's strictly non-commercial, not sign anything. I just put it up as a public service, as a labor of love as a tribute to my grandmother, whose own life was saved with evidence-based nutrition.