Okay, so now we will continue with the next session. Again, everything is about acquisition, and when we talk about flow and clinical flow, then obviously Doppler imaging is one of the ways that we can get information on velocities and we can do it in vivo. And it's my pleasure to introduce Alberto Gomez, who is working in King's College, and he's one of the engineers really working in the hospital and is really closely working with physicians and has a lot of experience on Doppler and also on how difficult it sometimes is to even just get imaging data out of the machine in order to do something with it. And so Alberto, please. Okay, thank you Bart for the introduction and thanks for the invitation. So as Bart will say, I'm going to be talking about flow and how you can use ultrasound in the clinic to measure flow and also how you can actually go further and do things that are not yet implemented into the clinic but hopefully will be to do more interesting stuff with flow. So the first thing I wanted to try to clarify is what actually is flow. So in preparation for this talk, I was talking with a friend who knows nothing about engineering and I asked him, can you send me a picture from your standpoint that summarizes or explains what flow is? And he sent me this. Okay, ice is broken now, so this is not what I was after but I thought it was funny so I just put it there. This is what I think of when I think of flow. So flow is something that can be really complex that is definitely 3D and varying over time in general. So we need a formal way of defining this mathematically if we want to do some engineering with it. So mathematically or more formally blood flow can be described as the motion of particles and if we are looking into cardiovascular flow it's the motion of blood particles throughout the cardiovascular system. So in terms of what we can measure and how we can mathematically characterize flow there are two quantities that I'm mostly interested in and actually we can derive one from the other. So the first one would be velocity, flow velocity and this is a vector field. You can think of it as a collection of arrows that vary over time and space, v there and essentially the direction of the arrow tells you the direction where particles are moving and the magnitude of the arrow is how fast the flow is moving. The other quantity that is very interesting particularly in the clinic is the volume rate or also called flow rate and this measures the amount of flow, the amount of particles that go through a surface per time unit. So before continuing I wanted to just clarify one thing about flow rate so if you consider this as an area Oh, sorry. Okay, sorry. Can you hear me now? Yes. Yes? All right. So flow rate is a global quantity over time, a scalar value over time and it's calculated as the amount of flow or the velocity of this flow that goes through a surface and something that is very important and we will come back to this in a second is that when you calculate this this measurement should be independent of what surface you choose. So in this case you have this surface but it could be any other tilted surface. The only thing that matters is if you define a vector perpendicular to the surface the only component of the velocity that influences or contributes to flow rate is the component parallel to this vector n. So velocity has three components but you only need one to calculate flow rate and we will see later on why this is important. Okay, so now that we have defined flow how do we measure flow? There is many methods and many techniques to estimate and measure flow. We have seen some this morning and we will see some more later on. These are some methods that are not or not very much used today or at least in the clinic anyways. I won't describe them but I just wanted to have them as a reference. On top of this there is also flow MRI which we will hear about later which is really impressive what people can do with it these days but today I'm going to talk about Doppler ultrasound which is arguably the most widely used method for measuring flow at least cardiovascular flow in vivo and hopefully throughout this talk I will convince you that this is a technique to use. So how does Doppler ultrasound work? Well you probably are familiar with ultrasound but I thought it would be important to cover a bit of how ultrasound is acquired because this will tell us about what are the limitations and what are the features of ultrasound that make it good for flow measurement. So very briefly ultrasound is acquired with an ultrasound transducer which looks like the picture on the right and for my diagrams I represent this as the diagram on the left is a representation of piezoelectric crystals these are the little squares on the surface and these crystals have the property that when excited with an electrical pulse will vibrate and will propagate a pressure wave into the tissue but also when they receive a pressure wave from the tissue they will create an electrical impulse. So this way we can send waves and receive waves. So for example if I excite only one element as in the figure the red one is the active one I'm doing a lot of simplifications and assumptions but for understanding I think it's okay a spherical wave will be generated at this element and will be propagated into the tissue so if you excite not one but two elements at the same time then two waves are generated and they interfere coherently, they sum up and then you have a different wave form. You can do this with many elements and it's of particular interest the case in which these elements are not all excited at the same time but you have some delay pattern so you see these ones go before and then the ones in the center go after this creates a parabolic delay pattern so the wave is actually focused on one point at a certain depth so you hopefully can see that the wave is converging here and then by changing these delays this wave can be made to converge before or even closer to the transducer and the reason why this is important is because when you have say a particle and you want to make a picture which shows where the particle is if you focus your wave adequately both laterally so you choose the right sort of line that goes by the particle then you will receive a strong echo back and then you can have a nice picture and this is also important because if instead of just one you have multiple targets as in this case by focusing your target adequately and your wave adequately and by choosing the line where you want to scan most of the signal will come from here so you won't capture this using this specific wave and this gives us lateral resolution so it will give you the ability to result for lateral direction which is this direction here so with this in mind the way we make an ultrasound image is just do this over and over so send the focus wave one line after the other so this is how the lines would cover the space and once you have gone through the entire image you end up with an image of the particles that you have as you see on the right hand side okay so this is how you make an image and in 2D and if you want to do in 3D you can as well with ultrasound and there are basically two ways of doing this so the first way is what you see on the left it's called mechanically steered transducers so essentially it's a 2D transuser that is steered with a little motor and you acquire a collection of 2D slices you put them together and you have a 3D image the other option is shown on the right is called face matrix arrays so this is instead of just having a line of elements you have a sort of chessboard of elements as depicted here and this allows you to choose your line not only on a plane but on 3D roughly so then you will end up with a 3D image as well and these two systems have different properties that will show from now on when I use 3D I'm using this technique okay the face based array so we have seen so far how you make an image but we are interested in flow so when you want to measure something dynamically changing such as flow you have to consider what is the frame rate of your acquisition system so let's try to figure out what is the frame rate that we can get with ultrasound and we can do so by putting together a few numbers for example so basically the limiting factor here is the speed of sound because we have to send an echo wait until the echo comes back and the echo travels at a certain speed which is the speed of sound this speed of sound depends on the tissue or on the material you are looking into and in human tissue is roughly 1450 meter per second perhaps a bit higher depending on the tissue but essentially if you want to go to see for example the heart in patients you normally go deep perhaps 18 centimeters so the time it takes to acquire just one line you have to multiply times the number of lines you are acquiring say 64 and this is just an example because you can go actually to a much higher number but if you do this then it takes about 16 milliseconds to acquire one single image and there are many techniques that can improve this a little bit and we will see some that can improve this actually a lot but for now let's assume that 16 milliseconds is a standard frame rate in a clinical kit so this allows you to get about 60 frames per second so now the natural question to ask is is this enough to look into flow so if it wasn't what am I doing here well it is not enough actually but yes we can do something with it the reason why this is not enough is for two reasons so the first reason is blood can move for example in the aortic outflow in a healthy patient it will be about 1.5 meter per second or even higher and in disease patients it can be actually much much higher so if you are at 60 frames per second and you are looking into a particle of blood this particle can move like 2.5 centimeters and we know from the previous talk that you can track perhaps displacements that are just a few pixels so this would be in standard Alzheimer's solution 40 pixels easily so of course you are not going to be able to track it so far plus the fact that blood is a 3D phenomena so if you are doing 2D ultrasound and you wait this long your particle for sure has gone out of plane and actually the other interesting thing is that we don't see blood in ultrasound so if you see this picture which shows so this is a persternal long axis view of the heart this is the left ventricle here this is the aortic valve and this is the mitral valve so the atrium here the blood goes this way to the ventricle and then gets ejected through the mitral valve also I made use of this image to sort of advertise how good ultrasound is in terms of resolution so if you want to do this with MRI good luck but the thing is that we don't really see blood and the reason for that is that the signal coming from tissue is so much stronger than blood that all we see is tissue the signal from blood we do and we use that to measure blood flow but in normal images we don't see it so how can we measure blood if all these problems are there well let's have a quick look at what happens when we try to make a picture of something that moves with ultrasound so first let's imagine that we are sending continuously a wave and this wave is being scattered by a particle that doesn't move so if this is the case then the frequency of the wave that we send is the same as the frequency of the wave that we receive and we are all good now if the particle is moving for example moving away from the transducer what happens is that the wave is getting compressed along the direction where the particle is moving and it's getting spread in the other direction so we will still receive some echo but it's like the particle is leaving the echo away behind so the frequency that we receive will be lower the wavelength that we receive will be higher and if we can then compare the frequency that we receive with the frequency that we send and they are different then we can figure out the velocity and as you know this is known as the Doppler effect so just one precision about the Doppler effect which also is quite important for what we are going to do now because we are just relying on waves being sort of compressed or spread due to motion when we receive them we have to take into account that we only notice or we only measure whether a particle is going away or towards us but we don't know with which exact direction and actually this is quite important because for example if the particle is moving perpendicular to the line along which we are measuring we won't notice any difference in the frequency so we will think that the particle is not moving at all so in other words we only measure the component of the velocity and this is illustrated here so for example if the velocity the flow is going to the right we are scanning with our ultrasound system along this direction what we measure M here is just the projection of this velocity along this line so depending on our angle of insonation we will measure a different value so then effectively what we measure is the true velocity magnitude weighted by the cosine of the angle the flow direction and the ultrasound beam direction and this is very important so if you haven't got or if you don't remember anything up to here at least remember this this is the one very important thing so far so moving on how do we use this in the clinical practice so there are two types of Doppler that we use one is continuous wave Doppler the other one is pulse wave Doppler you probably have heard about this already but I would actually say that pulse wave Doppler is what we normally use and the difference between the two is continuous wave Doppler is sending a wave constantly so when you receive an echo you don't know when this echo was sent because they are being sent all the time and because you don't know when it was sent you don't know how long it took or how deep it went so you cannot resolve in depth you don't know basically where the echo was generated so if you want to locate in space the echo in a pulsed way so that you know when this echo was sent and as a result instead of receiving a continuous wave you basically receive a burst of pulses and when you try to mix it with your reference signal to figure out what is the phase shift you end up with a signal that looks a bit like this so essentially your goal is to figure out the frequency of the signal and each of the pulses that you sent is just one sample in the signal so by the Nyquist theorem it is at the most half the frequency of your pulses of the pulses that you receive what this means basically is that the maximum velocity that we can measure with a pulse wave Doppler is limited by this phenomenon by this pulse repetition frequency and therefore there is a maximum velocity we can measure and beyond that there is aliasing and more graphically what that means is just think for a moment that this is your wave this is the velocity over time and we have this velocity that we can measure then we measure it as it is of course with the angle compensation thing but it can be that the maximum velocity we can measure with our system is lower than the maximum actual velocity and then aliasing occurs and this is captured as folding of the wave so the positive values go back to negative and so on and this is what we measure and this actually is a problem it is always present or often present because for example the maximum velocity we can measure normally in the clinic with a Doppler ultrasound transducer will be of the order as well of 1.5 m per second which is of the order of the velocities that we are sometimes interested in measuring so this is not exceptional this happens very often and even more so with disease patients but the good thing is that we can locate in space like I said before where the velocity came from and because we can locate in space where the velocity came from we can actually draw a map of velocities and this is called color Doppler imaging which looks like this so this is using similar basically pulse wave Doppler but in all points of the space within this area here, this color box and then we just give a color to the velocity that we measure blue if it is going away from the transducer or red if it is going towards the transducer and this is an example of aliasing happening in color Doppler imaging so in this case the flow is going down it's not that it changes directions here suddenly it's just because the velocity is so high higher than the maximum velocity we can measure it appears with opposite sign so we just have clinicians normally are aware of this and just keep this into account for their measurements and this is just an example of this being done over time okay so this is how ultrasound works how Doppler ultrasound works how do we use this to actually measure quantify flow so I'm going to cover a few standard techniques so what people actually use in the clinic and then I'll move on to perhaps more interesting stuff, more emerging more research oriented things so in the clinic basically color Doppler imaging that I just saw is very rarely used or if it is used it's used more qualitatively and have an idea of what flow looks like but clinicians in general don't draw any numbers out of it there are some exceptions but I'm always talking in general so what they use is this pulse wave Doppler or continuous wave Doppler depending to quantify flow in specific areas normally in vessels in hard inlets or outlets and actually it's not much they measure with this in summary it would be the maximum velocity or the peak velocity then they use a simplified version of the Bernoulli equation which basically translates peak velocity into pressure drop and then actually often times when they talk to each other they talk about peak velocity and pressure drop indistinctly as if it was the same thing it means the same thing from a clinical perspective to them and actually most of times they don't even compensate for the angle they just try to align the beam with the vessel direction and that's it other things that they do is also more qualitatively is look at waveform pattern so this would be the typical inflow pattern so you have an active phase or a passive phase in the ventricular feeling and then an active phase so first velocity of blood coming into the ventricles is higher and then a bit lower in this wave with a sort of atlas they have of what these waves should look like or could look like for different diseases they can make some assessment of cardiac condition and then for the more adventurous clinicians they would go and do some more advanced measurements which basically is just drawing a line on the waveform and then they not only have the peak velocity but the actual average velocity over the entire cardiac cycle and this can help them estimate for example average gradients at their region and sometimes they even measure the diameter of a vessel and then assuming it's circular they can calculate flow rate assuming that the profile of the velocity is parabolic and so on and so forth and that's pretty much it so this probably covers 90 plus percent of all the clinical assessment that is done routinely using Doppler ultrasound and I think this is important to know because you know we sometimes as engineers develop things that are rocket science but it has to be used to be useful, if it's not used it's quite useless so I think the key to making something that can be used is it has to be easy to use, fast or at least give an answer in clinically compatible times and it has to not require an engineer to be there all the time because then it won't be implemented widely so that said I'm going to talk now about things that are a bit more exciting this from an engineer perspective than this so the first thing I'm going to cover is how you make measurements and quantification which does not depend on the angle so of course the first thing you can do is just compensate for the angle yourself so you estimate what is the direction of the flow by thinking or assuming that it goes following the vessel which is a reasonable assumption to make and then you just divide whatever you measure by the cosine of this angle between the vessel and the transuser and then this will have an amplifying effect in the velocity that you measure so this is the simplest compensation you can do and most machines have this built in so you can do it in the clinic now a bit more advanced than this is what I said at the beginning so if you want to estimate flow rates you don't really need all the components of the velocity you only need one which is the component perpendicular to the surface you want to measure so if you have 2D color Doppler for example your velocity will be measured along these green arrows so actually it is absolutely impossible to have a surface that includes the entire vessel and is perpendicular to this point because it would be contained in the vessel so what you can do is you can use 3D Doppler so if you have your ultrasound field of view like this cut in the vessel so you have spherical caps which are surfaces like this and the velocity that you measure will be perpendicular to this sphere or actually along the radius if you want so you can use this sphere to calculate flow and then you have the component of the velocity that is parallel to the perpendicular vector of surface and you can do angle independent flow quantification so this is some work we did in the past using this technique so placing a spherical cap this is what a color Doppler image in 3D looks like and this can give you some accurate results this also has been taken over by some vendors so not my actual work but similar work and this is examples of clinically usable systems that are available in some other machines and of course you have to select your slice or your spherical cap sometimes this can be done using computational models that automatically fit the images that you are taking and so on and so forth to be honest I don't know any clinician that uses this routinely but it is available so this is about flow quantification angle independent and then now the other thing about measuring block flow that I think is even more interesting is how to measure velocity fields and we have seen examples of this yesterday also this morning with optical imaging and the idea is that you can measure the actual velocity at every point in a plane or in the space if you do some tricks with your Doppler imaging so just some fundamental ideas behind all the methods that I will show so the first thing Doppler ultrasound can measure one component of the velocity so it follows that if we can do many measurements of ultrasound then we can measure many components and if we can put them together we might be able to recover the whole velocity field and this is normally called vector flow imaging or vector flow mapping so the goal here is to find what are the missing components that we cannot measure with Doppler with the standard Doppler at least and then apart from trying to imaging from different views to get different components you can do things like trying to trick your ultrasound system and make it able to detect Doppler shift laterally or you can try to incorporate some physics or some modeling into the reconstruction so that you can constrain your data with a model of how flow should behave to get the missing components or you can actually do particle tracking and this is what was mentioned before echo PIV which actually is not PIV but it's a bit similar so because PIV was established so people thought using the same name could help publicizing it a bit more I suppose so it does track some particles so it's fair enough that's another method so I will first try to cover what you can do with standard kit and then we'll move out to something that you can do with non-standard kit so the first one of the first approaches to estimate velocity fields in 2D the first obvious application I think has always been the carotid artery do you think back of the keynote speaker on Monday so she was talking about how in bifurcations you can have some issues with clots or thrombos that can appear and that there is a tight relation between the risk in the patient and how flow is affecting that area so of course it's very important to measure flow here and this particular work consisted of basically taking ultrasound images double ultrasound of this region from two different angles and then reconstructing by triangulation the resulting vectors and this is proof that you can do this and for mathematical stability you can use some regularization based on the physics behind flow but it's done with standard to the ultrasound but you need to images so you need to either move the probe because you cannot put two transducers at the same time the waves would interfere so you have to move the probe so it can be a complex problem another alternative is transverse oscillation so this is a technique which is also actively investigated nowadays and the idea it's basically that you can change the way your wave is acquired or your image is acquired so that you will be sensitive to motion on laterally so if you remember from what I said about the Doppler effect you can only measure velocity that goes along this direction and basically this is because your point spread function has some oscillations or is a pressure wave so it is oscillating as the wave propagates it doesn't oscillate laterally so there is no frequency shift laterally but if you can make your wave to oscillate laterally then in theory you can measure this component of velocity as well so the way you can do this by something called appetization so appetization consists of basically waiting the intensity of the pulses that come out so in this case this curve this red curve here shows that you are giving more weight to the signal coming from this element perhaps zero weight here and then more weight here and very similar to what we hear before in ultrasound if you're in the far field so basically for our interest anywhere in your image domain further a few centimeters from the surface the point spread function that you will get is the Fourier transform of your appetization function so if your appetization looks like this so like two peaks more or less then your Fourier transform will have actually will be a sinus so it will be an oscillation and then the bigger your aperture the higher frequency this can have okay if you want you can think of this intuitively in a different way so what we are doing here is exciting these elements these elements and not these elements so it's like if you had two transducers one here and one here so you are measuring from two directions and then you are resolving a vector so what is happening is what I said first this is just the intuition behind it and also I think it's useful this intuition because it helps you figure out what is the limitation of this method you can do this for surface vessels if you go deeper then the angle between these two beams gets very very small and then your two views are less and less independent from each other so the accuracy will decrease and you are limited by how big your aperture is but you know people can do this in the for example in the carotid arteries and you can resolve your flow and then you are not limited you recover two components with only one transducer other things you can do is something called echo dynamography this and the next technique are very tightly related so we heard yesterday something about splitting flows into two flow fields one which is vortex only and other one which is the laminar component and when you sum the two you have the flow that you are measuring so this technique basically aims but that's a lot of assumptions for example it assumes that this vortex is symmetric and then by using which perhaps is fair enough in many cases so using this assumption you can figure out the components that you don't have in this flow and then for the laminar flow you just assume that there is no through plane flow so using this continuity equation pretty much in the same way we saw before you can resolve this and then you sum them up again and you end up with your flow field so it will look a bit like this and I personally don't think this is particularly accurate but it has the advantage that it only needs the Doppler data and also it has been already implemented in some commercial systems and I'm not sure how widely used this is but it does exist now actually more use than this and partly related is the work by Dr. Vermejo who was speaking yesterday and Damien Garcia which is you can call this just vector flow imaging and I won't get too much into the details because it was explained yesterday but the idea is that if you can combine the motion of the wall that you can track from anatomical images with the one component that you get in Doppler ultrasound you can use the continuity equation to recover flow and the idea is that the way the wall moves constrains the way flow moves so because your flow is incompressible or you assume it is incompressible if you think of it as the wall for example when it moves away from the fluid the fluid has to follow otherwise there will be void so the fluid has to follow or if the wall pushes the flow along this direction then the flow also has to be displaced it cannot permeate through the wall and so what that means is that the component of the velocity perpendicular to the wall has to be the same both in the flow and in the wall you can measure it from the wall and you can review the initial condition for this transverse velocity and the other component of the velocity you have it from Doppler so solving this equation you get this flow and then you can do all the fantastic things that Dr. Bermejo was showing yesterday so this is in 2D and then last is the famous ECO-PIV so this essentially is just injecting some tracers some particles which are in the bloodstream and then these bubbles will be shiny and we will be able to distinguish them in a 2D image and then we can track them and then if you do that then you can recover vortices and visualize them in different ways but essentially you need a contrast agent so that's the main disadvantage and then of course you still are not solving these problems of particles going out of plane if they do you can do some things in 3D as well and this is some work we did a few years ago so the idea is if you can do 3D color Doppler then you can have one component in a volume not only in one plane and by combining at least three different views each one will contribute with one component and when you align them together these components will be different and then combining this with wall motion you can get intraverticular flow in 3D and then you repeat this over time and then your flow will be 4D so you can do this without using the wall motion as well but if you do that you need to have perfect overlap of all the Doppler views over the entire ventricle and this is very challenging because basically your frame rate in Doppler because you need to send many pulses is less than in a normal anatomical image and additionally if you want to keep up with a decent frame rate then you have to narrow your field of view and even more so in 3D so normally your images are quite thin so here you can compare the size of an anatomical image and a Doppler image so it will only be a fraction of the size of this one and these figures here represent overlaps so this is the ideal case white is where the three views overlap and then the different shades of gray mean overlap of only two views or just one view and sometimes even no view at all so this is the ideal case this actually is taken from real clinical cases there are bits of the ventricle that are not covered and if you try to reconstruct blood flow this is just a 2D slice of the reconstructed flow well you will have lots of errors so in the first row it's just using the Doppler data but then if you incorporate wall motion to constrain the problem you can actually find your vortices again with more or less accuracy but at least the shape of your flow is correct so we did this and then we tried to see what this can give us so this is an example of visualization in this case we reconstructed flow just in the tunnel or in the channel that goes from the atrium to the ventricle so this would be in this case it's the tricuspid valve because it's a specific case of disease patients where the systemic ventricle is the right ventricle and basically just for visualization we seeded some particles in the atrium and so that they were going towards the ventricle this is not of clinical significance this particular example to see whether our results were somehow consistent and then the flow is not reconstructed here so it's meaningless what the particles are doing after the valve we did this as well in the whole ventricle in pediatric patients so this is a case of a pediatric patient these and these represent the same field this is just a 2D slice of the flow getting into the ventricle and being ejected and this is trying to have a 3D visualization of the vortex so this was using streamlines along the velocity field in 3D and then we wanted to do something a bit more meaningful clinically so we tried to see whether we could establish differences in the vortex shape between different patient groups so these are all systemic right ventricle patients a hypoplastic left heart and some of these patients have left ventricle, right ventricle that in shape looks a bit like an ellipsoid so more or less similar to what a normal left ventricle would look like but some of these patients because the left ventricle grows a little bit then the right ventricle is elongated and has this asymmetric shape so by using this technique we reconstructed vortices in 2 different, well in I think 5 or 6 patients this is just 2 examples so what we show this is the cardiac cycle goes so we saw that the vortex was forming more or less in a normal way in the patients with this shape of ventricle but in patients where the ventricle was elongated the vortex only formed on one side and actually this is highly inefficient and these patients both actually will need some surgery but then using these techniques can help clinicians decide what surgery would be best to correct for these issues so then some limitations of these methods so one main limitation is frame rate so I said that well 60 frames per second can be good Doppler ultrasound pulse wave Doppler and color Doppler imaging have less frame rate because you have to send many pulses so then you are already being a bit compromised and then you have to start acquiring over many cycles and other aging and so on so we we heard yesterday that for the studies that Dr. Bermel was doing he was acquiring many cycles and then interleaving to get sufficient frame rate and ideally you want to not have to do that also 3D methods are very very limited both in frame rate and also in resolution in accuracy and so yeah and then there are a number of issues you have to calculate wall tracking and so on so this is all very nice but can we do better and this is where I think things get really really interesting and this is where I introduce ultrafast ultrasound and I think this is really the future of ultrasound because it can help overcome many of the limitations that we have now it enables very very fast acquisition and this then solves at least partly the issues we have with frame rate with Doppler the maximum velocity we can measure with Doppler and enables a whole new thing set of things that we couldn't do before so I'm going to show a few examples following the same trend as before so first 2D carotids and moving on into intracardic flows so how does ultrafast ultrasound work actually surprisingly or perhaps not is simpler in concept than standard ultrasound so basically the way it works is instead of focusing our wave we actually excite well this is one way this can work the main idea behind is that instead of sending a wave that is focused in one direction a wave that will sound the whole space so all the things that you want to look at will swap or invaded by the sound wave and then you will receive data from everywhere at once and one way to achieve this is by sending a plane wave from the transducer which you can achieve by setting the same delay in all elements for example and then this wave will propagate and will touch all the scatters if you want in your space and then if you do this then you don't have to go line by line you can do just one line for everything so as you see it can be simpler so if it's simpler why weren't we doing this before well I guess you lose well you need to come up with a way to find your lateral resolution to start with but also if you have to keep up with the frame rate and reconstruct your image in real time you need a very very high computational power and this wasn't available to us until recently with the advent of GPUs and so on so how fast can we make pictures if we use this technique well it's basically using the same calculation we did before but just dividing by the number of lines because now we only need one line using the same numbers you would get 4,000 frames per second and this is not a typo it's really 4,000 frames per second and actually this is if you want to go all the way to 18 centimeters if you want to do carotid which is just a few centimeters down you can go to 10,000 or even 15,000 frames per second so Ultrasonic already was the fastest modality at least compared to MRI and so on and now I actually doubt that MRI will ever catch up to this this is very fast so if you would acquire a standard just to give you an idea of how fast this is if you take a standard ultrasound image say 60 frames and you play it with all the frames you would have using this a single hard bit would take a couple of minutes to reproduce so you can imagine the slow motion that would generate it's really fast so what can we do with this to measure flows well one of the first methods that used this was this one here where basically because we can measure with such a high frame rate we actually don't need to do the Doppler anymore we can track blood because the displacement is now very very small and if you do that then you measure both components at the same time so there you go you have a highly time resolved with high spatial resolution description of the flow in the carotid so these techniques were improved of course and then you have simultaneous tracking of the wall and the carotid bifurcation so in this case instead of just sending a plane wave and tracking particles this was done or tracking the blood sorry this was done using Doppler but because you can tilt the wave that you send you can tilt it in two different directions so it's like having two different Doppler views and combining them to calculate your vectors you get this but you are not limited to only two angles you can send as many waves as you want and you can send a few from different angles and then of course one wave and then it's not only a plane but actually the whole space so in theory you don't lose temporal resolution when you go to 3D so it's as fast in 3D as in 2D in reality this is not really the case because your signal to noise and your lateral resolution is low so you need to send more waves but there are people doing ultra-fast ultrasound in the heart nowadays in 3D at a few hundred volumes per second so it's still very very fast it's still from these groups here where they got this data and actually if you look at the literature on this we have lots of papers which are just 2016, 2017 so this is a very very hot topic now and if you combine this with a fancy visualization then you can see flows in the carotid in an impressive way like this so this is a normal subject and this is a disease subject this is actually slow down quite a lot so that you can appreciate the flow but you can see in this case actually there is a clot here so the flow cannot pass by properly it accelerates abnormally it will produce a lot of stress in the upper wall but if you don't have that the flow will be good and this is measured with ultrasound so I hope you agree that this is actually very nice and if you can have this in the clinic of course it will be surely for sure so this would be very useful so now moving on to other applications or to intracardiac there is particularly the work by Professor Lofstadge and his group in Trondheim they are doing a lot of work in particularly in pediatric imaging where they try to get flow maps in the children hearts and this is an example of what you can do with ultrasound in this case it's through a defect I believe this is in the ventricle so in between the two ventricles there is a little hole and it's quite important to measure what flow is going through that and that's actually very challenging but thanks to these techniques you can do that and this is another example where they use the same technique and now this is moving on to trying to combine such a big frame rate and doing that well you can to start with get really nice pictures of motion so you can track the wall and you can track the speckles of blood unfortunately we cannot see very well but you do see the speckles of blood so when you reconstruct this and you put some colors as if it was color Doppler imaging just to show the direction you get this sort of visualization where you can see now so much more nicely the particle the flow represented using this particle tracking methods and you can even go to the ventricle and get really highly defined vortices and basically dynamic images of intracardial flows and this is on a real subject a patriotic subject in the heart and I think it's difficult to think this is going to get cooler than this but it will because now the research is going towards 3D imaging of vortices this is not going to be the solution for everything ultrasound has many limitations particularly for artifacts that you cannot get rid of such as shadowing view dependent artifacts and so on but there is lots coming up and it's very promising as a measurement tool so we're getting to my last slide so I just want to do a summary basically ultrasound is great, it's high frame rate standard ultrasound you can acquire data in real time in bedside you don't need any fancy processing to get just simple flow measurements it has some limitations such as angle dependence aliasing and so on but it's very widely used so any small improvement you can make in ultrasound can improve potentially the life of many more people than perhaps in any other imaging modality at least for cardiac care advanced ultrasound particularly ultra-fast ultrasound and the combination of structure, anatomy and flow measurements can actually unveil a new world of new measurements that we can make for clinical use or to fit models and just wanted to say at last that it's 2017 and I have got away with doing a talk in medical imaging without saying machine learning so let's see if next year I can do the same so that's it thank you so much I'd like to acknowledge people at Kings that work with me at the Trondheim National University Imperial College London and I'd like to use one minute just to say a quick thing about Professor Peter Wells he's a researcher in London as well and he unfortunately passed away a couple of weeks ago so Peter Wells has a massive contribution to ultrasound in general but he's sort of the grandfather of Pulse Wave Doppler so I think nothing of what I said today actually would have happened without Peter Wells or definitely would have taken much longer to happen so I think it's right to acknowledge people who did such huge contributions to science and just with that I'm happy to take any questions that you may have thank you very much and indeed as you said like ultrasound is the bread and butter of cardiology and you see especially Doppler is quite useful and if you see what's indeed going on currently in research in these high frame rates it's extremely promising so maybe we just don't know yet what we can do with ultrasound so it's indeed a field which I think is very very important to look at maybe just very very briefly what's your idea is like when you do this high frame rate imaging everybody seems to jump onto the color Doppler in some ways and trying to improve color Doppler, improve the visualization but what about improving pulse Doppler for example or doing it making like at the same time acquisition of Dopplers from different valves in kind of different planes or whatever so I think that's a very good point and it's totally true so there are a few aspects I think that are important to consider so the first one is clinicians like what they use and what they are used to using so they like to see colors for flow quantification to the point that actually we had a clinician that wanted us to post-process flow MRI data to display it as colors because it was used to color Doppler imaging so that's perhaps one of the reasons why some effort has been driven into that but then I think one of the potential big impacts of this technique different to Doppler imaging is valve imaging so it's very challenging to look at valves because they are very small, they move very quickly and there's a lot we don't know about them so perhaps using ultrasonic ultrasound for valves can enable new measurements that we don't have now so I think also because ultrasound is something that you use in real time so people look at it and are already making a diagnosis as they are scanning it seems like it might not have such a big impact in the clinic during the scan because you cannot see at this high frame rates so I think possibly it will be sort of food for post-processing or for retrospective analysis of cases but again that's up to the future to decide, we don't know yet and what's your opinion about the use of for example Doppler imaging together with simulations for regularization or as input for one of the other yeah, so I think one of the problems with Doppler is that it is noisy and it's not so accurate or not so sensitive sometimes so most of the techniques I have shown require some sort of regularization and this regularization is mostly based on models so I think I think the big conflict between the two and at least this is what I feel when I work with modelers is that modelers normally have very clear what sort of inputs they need for the models so for example I have just as an example I have this case where we're trying to build cardiovascular or cardiac models with particular flow and the modelers would ask me for the velocity in the inlet of the heart and in the outlet to make the model and I was saying but I have measurements everywhere in the ventricle why don't you get this which is much more data and then you can build a better model and I was like no, the model doesn't work like this I need the inlet and the outlet and they will solve the equation so we argued a lot about this in the end the modeler won but I think that's where we should go to to make models that are more flexible in the data they need to take and that can take more data than they need to make something closer to the data so sort of use the model to regularize the data rather than the other way around and also with models it's like regularly people try to just model one heartbeat but when you see it's from beat to beat it's like one of the strengths of ultrasound is that it's real time as compared to MRI for example and you can really look at variations of beat to beat regular heart rates and things like that and how do you see that to use of information from this then to try to use that to also again improve the models or explain some of the things yeah so it's true that some techniques assume that your cardiac beat is periodic and then you can average over many cycles and ultrasound doesn't need to do particularly ultrasound doesn't need to do these assumptions and so I think there is place for investigating particularly changes in cardiac beat or cardiac rhythm and their exercise or their irregular beats and so on the only problem with ultrasound and we haven't solved it yet is that ultrasound cannot see through bones or through air and the heart is covered by bones and air so it's difficult to look into the heart and so you have to exhale and so on so it's difficult if you want to do exercise tests where people are running for example and you want to do ultrasound during that it can be complicated but I think we might end up finding a way and ultrasound would be the ideal technique for such dynamic imaging the question there thank you for your presentation I have a question about ultrafast imaging it has been some years that the technique has emerged actually quite some years now and it has not translated to clinical not what I heard what is the biggest slowing down is it money because it costs a lot of money or are people reluctant to use it so at the risk of being maybe chased after afterwards I think manufacturers play a big role in this and they already have a really good and established business with standard machines and because we still don't know what direct clinical benefit ultrasound ultrafast ultrasound will bring in it's not worth reinventing or pioneering all the machines to include that so most of the machines where people do these things are like smaller companies or research prototypes that said I think it's just a cycle of industry so people are already doing ultrafast ultrasound with big manufacturers in the research labs but that naturally will take perhaps six, seven years until it makes it into a product and it's just the cycle of industry so I don't think it's slowing down I think it will come up eventually it's just these two factors other questions maybe also when we're talking about companies maybe I think it's worthwhile if you share us a little bit of your experience in the beginning to try to work with this type of imaging data because I think a lot of us would struggle with that as you say before it's like if you want to do research on the imaging or a different reconstruction of the imaging you need to get to the data and as you say like with a high frame rate indeed there's a couple of experimental systems that if you have enough ground money you can try to get it but in order to get things out of the clinical ones and then to see how the interaction with companies goes I think it's an interesting experience that I know that you have so maybe you can explain it slightly or shortly too one thing to know about ultrasound the widespread in the clinic is 30 years or even more but it has always been a real-time imaging modality so at the time with the computers that were available it was really challenging to do this real-time imaging and rendering and so on so companies had all sort of patents and IP protection over how they did that because that actually made the difference between a machine that can be used and a machine that cannot be used so I think as an inheritance of that ultrasound companies have been more protective perhaps than MRI or CT company well I don't know about CT but definitely MRI so most manufacturers have the data stored in a way that you can only see with their own software and this can be frustrating for ultrasound so in my personal experience I had to sort of do reverse engineering many times to try to figure out how data was stored and try to pull it out and spend lots and lots of energy at the same time trying to chase after people in companies to make agreements to have access to the data and so on so I don't think that has changed in the last 10 years unfortunately but if you have a good agreement with a company and they are happy to collaborate with you then you can have access so the access is there is not that it's not available it's available but not given to anyone unfortunately and that's a fact I think it's not that we probably it's not fair to blame only the companies for this but they surely play a big role now that new prototype machines are coming out and in a very open way I think this is making companies rethink how they deal with this data so hopefully for you it would be less of a problem that it was for me but I think if you're going to start a new project in ultrasound before you start make sure that you will have access to the data otherwise it can cause a big pain and delay and maybe also like you really work in the hospital how is the interaction with the cardiologist going on is it you that present let's try this or do they come with a problem that you try to solve it's like what's the way to do this type of research in close collaboration so I think in my case you have to spend a lot of time in the clinic lucky enough to be an engineering department based in a hospital so we have lots of access to clinicians and we go to the clinics and we are there while they do exams so this allows us already to see what things could be improved and then I think you really have to listen to clinicians because we have different concepts of what is something cool and important and an example of this is engineers like all this visualization where you see vortices and things going from one place to another but most clinicians want something that tells yes or no like do I have to do surgery yes or no I don't care what the vortex is swirling or if you are using particle tracking or streamlines or path lines or whatever so it's very important to understand that what clinicians want is not where you want but that whatever you do can be useless if clinicians don't like it because then they won't use it so what we do is we make sure we have a system that can get the data quickly from the machine and that we can process this data quickly and then the best is to just show anything you have regardless of whether it's very preliminary to clinicians as soon as you can and then it's a closed loop feedback and that way you can come up with something that is useful also I find it very useful to go to clinical conferences as well just to see what maybe what they think is more important and normally you see techniques that were shown in technical conferences two or three or four years ago it's the time it takes to make it to something clinically useful so I will definitely recommend you do that if you can we're still waiting for the next presentation to be ready so we have still a bit of time to discuss or if there's other questions I like your comment about machine learning and I wanted to ask you what is your feeling about deep learning and consider all the advances that has been done recently especially in the research process I think it's a very important question and even more so for ultrasound the one thing to always keep in mind about deep learning in particular is that you normally need a huge amount of data and basically it consists of learning from a huge amount of data hopefully label data but maybe not necessarily and ultrasound is ideal for that because you can generate a lot of data so we have a clinic actually for fetal imaging it's not cardiac imaging but it's fetal and we acquire data in real time with a normal system it's not an ultra fast ultrasound but because we are constantly acquiring all the data during an hour or so we end up with about 20, 30, 40 gigabytes of images for each patient so actually no so it's 20 gigabytes of images about 40,000 images for one patient so 40,000 images is a lot of data it's not data of a group or representing variability within a group but it's for one specific patient but it's something that normal ultrasound systems can do so I think if you have a way of exploiting that from machine learning perspective that has a huge potential then apart from that the other thing with ultrasound is it's a real time modality that things are done in the clinic so if you want to do machine learning based on annotated data it gets a bit complicated because then you need that people annotate the data as it is acquired people do take some annotations such as measurements or labeling different organs and so on so you can use that but if you want things like segmentations and so on then it's actually a problem not only in ultrasound in medical imaging and this is why we are lagging with respect to computer vision there is not so much annotated data yet and the data that there is is not publicly available so easily because it's a bit sensitive data from patients and so on so I think once we solve these two issues and we at King's together with Imperias we are working for example things like how do they call this crowd labeling or something which basically is making available data or parts of data for everyone just to make sure that everyone can label a bit so that in the end a lot of data will be labeled I guess this is like tagging people in Facebook so if many people tag many people in Facebook then you have a huge database so yeah you need these two things for using this deep learning in ultrasound I think also talking about this you said in the beginning that when still most used measurements for images that are taken like the pulse stoppers from valves and things like that and it's especially looking at patterns it's like currently in clinical practice they would just do two measurements and they try to learn from the pattern whatever they see based on their experience but isn't this also like a kind of potential application for more like machine learning whatever type it would be yeah absolutely so I guess the way I see this is if you can sort of translate into a machine the techniques that people apply to data based on what they have learned from books or from universities so if you can train a machine to do those things then you can use machine learning for that and it doesn't need to be necessarily deep learning but it can be deep learning the other very interesting thing about deep learning is that also some work we have been doing when if you can use for example convolutional neural networks to do segmentation on data not only you will get segmentation but you will get patterns of how does the network recognize your image or in other words you will locally see what parts of the image have made the network for example classify your image or carry out your segmentation and this basically will give you an idea of what makes the machine think that this is a four chamber view for example and actually that correlates quite well with anatomy so this could be used as well basically trying to dig into the networks and see not see them as a black box but rather as something that is mapping the image in some way and then you can learn you can do all fancy things with that as well so you can compare the 4D ultrasound versus 4D MRI for a segmentation yeah so actually I have and the result is that they are very difficult to compare and there is no ground truth so it's very difficult to know which one is better so maybe I can give you the conclusions I got to so the first conclusion is if you want to compare them just side by side so to say you will notice that MRI visually is much more pleasant it looks much better partly because it's smooth and nice and I'm not totally convinced that this means also that it's more accurate or at least not for this reason I think in general it probably is more accurate than 4D flow from echo so this is one thing the other thing is that you are measuring slightly different things so with MR measuring displacement based on phase shift or phase contrast of the spin as the particles move and with ultrasound you are looking at waves and that have a history so they have propagated to the transuser in a different way so there will be artifacts affecting how they are captured so that's one thing the other thing is if you want to compare them point wise so this vector with this vector and so on over the entire image which might be more interesting then you have to solve first the problem of aligning MR and ultrasound and that is very complicated so we have some work done earlier this year where we try to do this just for anatomical comparison not for flow and after using lots of methods that are available there we found out the most reliable way of aligning ultrasound and MRI is manually so just by picking landmarks with a clinician and that just works so good or the other methods work so badly except for a few cases that people use to publish papers but can you delete this from the video no it's a difficult task and so then the third thing is if you want to compare 40 flow in MRI and ultrasound on real patients then you have to acquire the data in the same conditions and this is very difficult so for starters in MRI people lying on their back in ultrasound you will be laying on your side then in MRI if it's for example small children they will be under general anesthesia so if you want to have the patients as well under general anesthesia in ultrasound well it's tricky but you normally won't get that then under anesthesia is well it's limited how representative the actual flow is then of course you have to acquire things on the same day so it's very difficult I think and then we did some work trying to do this comparison on phantom data because I think that's the proper way to do it because it will solve all these issues and it's really difficult to come up with a phantom that you can see both in ultrasound and MRI so we have tried quite hard and not been very very successful yet but yeah it's an open question so the short answer is they are a bit different and probably Doppler is still not as accurate as MRI maybe a last question we almost ready with the next talk is like what you presented and also where you're working is very much related to pediatric cardiology is there any reason why for this type of kind of things you see in the literature predominantly pediatric cardiology rather than adult cardiology which should be much bigger is that true? yeah actually that's a very good point well I so it's true most of my work is in pediatrics actually my exposure to adult data is more with volunteers when we try methods and generally pediatric data in ultrasound is of better quality because they have less fat the heart is less deep so there's less attenuation things that are easier to contain ultrasound first because it's smaller hearts and that I think that generally makes it easier to work in ultrasound with pediatrics plus the fact that it is more difficult to work in MRI with pediatrics because you need to do general anesthesia on patients the fact that the heart is smaller is a problem in MRI so it's like a good complementary situation between the two techniques so I think perhaps the fact or the reason well also it's very difficult to do in some patients in adults particularly big patients where you have a big layer of fat or the heart is very deep or they have big lungs because then you have lots of artifacts so generally if you can make a technique work in adults it will also work in children imagine wise with ultrasound this is not necessarily true so that probably is something to take into account and the same goes of course ultrasound and Doppler ultrasound or anatomical ultrasound yeah okay thank you very much for the extensive questioning I think it's really important also for the students to know what all this involves it's like really it's very interesting to see this kind of images and especially ultrasound a lot is going on but there's also a lot behind it and a lot of these things is like you really have to try to collaborate with a medical doctor see which is the field you can get you have to fight with the companies and things like that so it's challenging but it's actually also really kind of fruitful and very rewarding okay thank you very much thank you


we have a director spotlight number one for the gamers channel our first full-time director is zack aka real psychs what do you see behind me to check out his channel click right here zack joined us at the very beginning in december 2013 when freedom just launched he is the first full-time director contributing to the gamers channel and if you want to contribute to the gamers channel just go visit youtube.com slash gamers and apply leave some comments be part of the community you will get noticed and you will be contacted send them a message these are all ways to get in touch with the gamers channel zack prepared a video featuring who he is answering questions about himself and i'm gonna play it right now take it away my friend hey guys how's it going i'm zack recycling known as real psychs around youtube and i've been doing youtube for one year almost one year it's absolutely crazy it feels like it was just yesterday i've learned so many amazing things and met so many amazing people on youtube but the one thing that i could never find is a network where i felt fit somebody you know where i wasn't just another number they added to the spreadsheet i was a person i was a human with a face to my channel i wasn't just another number and with freedom they offered that so as soon as i got word about this absolutely amazing network i joined at the very beginning way back in december i joined it i could not let that opportunity pass up so i joined freedom and ever since then i contacted george a million ways and asked him george is there any possibly thing i could do whether it's voiceovers videos you name it and i wanted to do it i wanted to provide a helping hand for being in this network and he was nice enough to offer me the opportunity to run a channel called youtube.com forward slash gamers now on that channel we offer the freedom family you guys ways to gain exposure shout outs upload your videos you guys name it and we do it on that channel we bring you you know montages the latest and greatest updates on news and gaming in the whole nine yards once again i'm zack recycling known as real psychs and george has opened a lot of doors for me i'm very grateful and thankful for that and we run a channel called gamers this is your host real psychs also known as zack recycling and i will talk to you fantastic people later bye bye


Welcome everyone. So, now we completed our study of the steepest descent method or more generally line search methods in the previous lecture. So, what we will now analyze is the basic issue of how fast these methods can be. So, for we will now discuss what the rates of convergence or the speed of convergence of algorithms. So, in order to do that we need to first have a notion or a definition of what we would call the rate of convergence. So, let me define that for you to begin with. So, the rate of convergence. So, let R k be any sequence of numbers that converge to R or R star. So, we say that the order of convergence, the order of convergence of R k to R star is the largest value of beta greater than equal to 0 such that this quantity, this limit which is greater than equal to 0, this limit is k tends to infinity is finite. So, what does this mean? It basically tells you what is the largest value.  So, that is effectively what this particular thing is telling you. So, for large enough k, the iterates behave in this kind of way. So, if beta is and to be precise here, I should be taking the limits. So, if beta is taken, it turns out to be 1, we say it is linear convergence, which means that the distance between R k and R star, which means the updated distance between where you want to be and where you currently are is roughly equal to a constant times your earlier distance R k minus R star.  But what we are asking is how much progress are we making relative to where we were and how much progress have we made in the new iteration relative to where we were in the previous iteration. And that is what is capturing being captured by the rate of convergence. Now, if beta equals 2, we say it is quadratic convergence and this is usually very, very good because if you can get, if your iterates can converge quadratically, that means you are making the progress you are making is much more than what you at each iteration, you are progressing much more than you had in the previous iteration. And if it turns out that beta is equal to 0, sorry, if it turns out that beta is greater than, sorry, not 0, you are not equal to 0, if it turns out that this is, if it is greater than 0, greater than 1 and if it turns out that beta is greater than 1 and this constant here and this constant is actually equal to 0. In that the constant here, sorry, the constant here is equal to 0, then we say it is super linear convergence. So, now let us to what we will do is we look at the rates of by looking to study the rates of the convergence, we will look at first the simplest form of a descent method which is the steepest descent method and that too we will study it only on first on quadratic functions. Because once you, it turns out that a lot of what we want to say can what we want to learn about the rates of convergence can be learned from just this. So, let us look at this function f which is half x transpose q x minus b transpose x and I am going to take q to be is symmetric and positive definite. The gradient of f you can evaluate is equal to q x minus b and x star is then the unique, let x star be the unique solution, x star which is the minimizer of f is the unique solution in the minimizer x star, the unique solution of q x equals b. Now the good thing about quadratics is that you can calculate a lot of these, you can calculate a lot of these things in closed form. So, in fact, when we are doing this since you are doing now, if you are doing steepest descent with line search, the actual step that minimizes the function along a certain search direction. So, the search direction for us is pk which is equal to minus gradient of xk. So, we can actually calculate f of xk minus alpha grad f of xk. So, this is equal to say remember f of xk plus alpha pk. So, we can actually calculate this in closed form, closed form we can minimize, find the alpha that minimizes this. It turns out alpha, so find alpha and said that as alpha k. So, it turns out alpha k is equal to gradient of f at xk transpose gradient of f at xk divided by transpose q times the same gradient of f at xk.  So, this actually gives us you can now substitute this in all of these expressions in terms of this expression that where the gradient is equal to q x minus b and you can actually get this in a much more explicit form where xk plus 1 can be written in terms of xk. Now, it turns out that you can also therefore using this then we can also calculate compute from here how x the distance between xk plus 1 and x star and the distance between or the difference distance between f of x minus f of x star. Let us write this as so, first let us define the following let us define norm q norm of x as x transpose q x. So, this is effectively as a kind of a skewed or a tilted norm with where I am taking the weighting matrix as q. And now we can it is very easy to show that from here that x minus x star square q sorry this squared this is the squared q norm this squared is actually nothing but f of x minus f of x star. And now using this we can derive the following theorem says the following now with when when the steepest when the steepest descent method is applied to the function f exact line search what do I mean by exact line search by exact line search I mean that is where we are finding alpha in closed form. So, this is let us let me write this here this is what is called exact line search. The what we did in the previous lecture where we were looking for alphas that satisfied the wolf condition those are what are called in exact line search because they are not we are not actually finding the exact minimum there we are just putting conditions that that that are terminating alpha must be satisfied. So, with exact line search then we what do we get then we find that x k plus 1 minus x star squared in q the q norm is less than equal to lambda n minus lambda 1 divided by lambda n plus lambda 1 whole squared times x k minus x star q norm where what are these lambdas these lambdas this is lambda greater than 0 greater than or dot dot greater than lambda n are the eigenvalues of q. So, these lambda 1 to lambda n they are the eigenvalues of q now where lambda 1 is the smallest eigenvalue lambda n is the largest eigenvalue. So, now what is this relation saying this relation this q norm squared is nothing but the difference between the function value and the optimal value of the function. So, the difference between so, if effectively this term here or this term here essentially is capturing something like an error the error that you have or the departure that you have from your optimal solution. So, the error it says is less than at iteration k plus 1 is less than equal to this constant times the error that you had at iteration k you can see what is this constant well this constant is this. So, this constant is something that depends on the eigenvalues of q.  So, remember this was a convex quadratic function because I assume that my q is symmetric and positive semi definite. So, if I look at the contours of q. So, let me draw this thing here for you. So, contours of f so, this would be let us say is the my outermost contour this is another contour inside this is a another contour inside this is another contour etc. So, if you do this here is what we get. So, you start from here go here and then you do this and then one goes say maybe I will draw one more contour in somewhere here in some intermediate contour that takes us here then another one takes us here and another one takes us here etc. So, you can see what the way from what I have drawn these this here these are the iterates that your algorithm has produced it is starting from this particular contour this is your x this is the point x 0 from here it goes in the direction of it goes in what is what is the steepest direction of steepest descent keeps going till it encounter until it can minimize the function along that direction that minimum gets attained here at this at the second point here. So, this is point x 0 this is point x 1 then it moves then it again goes in the direction where the this is minimized where the it goes in the direction of the negative gradient minimizes the function along that particular direction gets reaches point x 2 then again goes in the direction where along the gradient now negative gradient at x 2 etc etc right. So, this is this is what happens in when you when you apply steepest descent steepest descent to this sort of quadratic problem. Now, here this was a very simple problem because we actually knew where the solution also where the solution was and you could have found the solution by simply inverting the matrix Q but you could have but we are we are trying to see what how the behavior of steepest descent method actually work. Now, what you can observe here is the main the main thing you can observe here is that the steepest descent method does tends to do this kind of zigzag it is it goes it keeps going zigzagging this way all the way towards the solution and the reason for that is this kind of zigzag is because of this Q that is that is introduced by the matrix by your Hessian matrix Q if the if the extent of zigzagging or really depends on how much you need to keep changing your directions of descent. And that itself depends on what depends on how the how your eigenvalues are how different your eigenvalues are. So, in the simplest case if all the as you in the simplest case what does this say well in the simplest case if all the eigenvalues are equal what would happen if all the eigenvalues are equal where in that case it would be Q would be all would essentially be would essentially be an identity matrix say all the eigenvalues are equal to 1 in that case what would happen then lambda n would be equal to sorry I should have put a weak inequality here my mistake in that case lambda n would be equal to lambda 1 and then this term would be 0 the term on the term in this inequality would be 0 and x k plus 1 would be exactly equal to would be exactly equal to x star. So, in that in the case when the eigenvalues are all equal the first step itself would not point in this sort of direction but rather take you point straight towards the towards the minimum the actual global minimum of the function it would point straight point you to this direction directly but because there is a skew in the function the shape of the function takes you in this sort of path it first the first you go in this direction then you go in this direction then you go in this direction then this way then this way then this way etc okay and the extent to which you will be end up you will end up zigzagging really depends on in general depends on the condition number or the ratio of the largest to the smallest eigenvalue right. So, as the as the condition number grows larger as the condition number grows larger the contours of the quadratic become more elongated they tend to sort of get more stretched out the contours become stretched out straight or elongated and more zigzagging occurs. Nonetheless one thing that this result actually tells us directly is that the steepest descent method converges linearly in general right. So, the convergence of the steepest method is linear. So, this sort of there is all this zigzagging but it does converge but it converges only linearly if your condition number is mild means close to 1 then the contours themselves would look more circular and the first try itself will get you close to the solution. So, that the actual number of iterations would be the actual number of iterations then would probably be a little lesser okay. So, here is the main thing then that the rate of convergence depends on the extent of curvature or the or how different the eigenvalues of the lowest and the largest eigenvalues are alright. So, this is what we are saying. So, let me make a note here. So, the steepest descent converges linearly on this on this sort of quadratic problem. Now, it turns out that for problems that are more non-linear where not necessarily quadratic some pretty much similar sort of result holds. So, I will just state again the theorem for you. You know, suppose f from Rn to R is twice continuously, twice continuously differentiable and that iterates generated by the steepest descent method converge to x star with exact let us say with exact line search converge to x star where if I look at the Hessian of f at x star this Hessian is positive definite. Now, let R be any scalar that satisfies this lambda n is a scalar that lies in this in this interval lambda n minus lambda 1 divided by lambda n plus lambda 1 to 1 where lambda n greater than equal to lambda 1 greater than 0 are the eigenvalues of the Hessian at x star.  Now, you can see how this why this is the case the f of x k minus f of x star is essentially the same as this error term and this is what appeared here. So, what is happening here is that you know as you for a if you are in if you are considering a twice differentiable function and you can look at the iterates that come from the steepest descent method with exact line search. And suppose they converge to an x star which where where the Hessian is positive definite then the function value or the error term is going down to 0 linearly with is having is showing linear convergence. This you can see and the rate the constant outside again depends depends on the depends on the condition number of the Hessian. The condition number of the Hessian at the point where the where the function converges. So, what is what is the lesson here the lesson here is one of course that the rate of convergence of steepest descent method is linear in the best case. And second is that the quadratic the study of the steepest descent method on the on a quadratic function gives you a good sense of what would be happening you know for a for a more general nonlinear function. And the reason for that the reason for that is basically the rate of convergence at the end of the day depends essentially on how the function how the iterates behave close to the solution because it is eventually a limiting quantity. And since the rate of convergence depends on and when you are close to the solution a quadratic approximation of the function is a fair approximation. So, quadratics tend to give you the performance of an algorithm on a quadratic tends to give you also how the algorithm would perform on more nonlinear type of functions.


Neil it's camny cool. So how are you mate?   I'm a flamingo. I know Good. Yeah, no no there does seem weird A good way a good way Yeah So if you didn't watch last week's video I'll put it somewhere up there in case you want to check it out But in a nutshell last week I took my BMC team machine into my local bike shop trilogy cycles here on the Sunshine Coast to get new cranks Installed which are seven point five millimetres shorter at the same time We cut the top off my chimney on the BMC team machine Which essentially made the front end of the BMC shorter because we cut the steerer tube off and the back end So where the seat is taller had to go up and down because what I neglected at the time Was the crank length change meant that I had to put the seat up and I was worried Had I wrecked my BMC team machine for my purposes anyway So after I edited that video and published last week I quickly jumped back on a phone call with Neil Stambry to get the full lowdown on the change in seat height And how to manage that and while on the phone with Neil He was explaining to me after watching my video that he actually thinks the chimney chop will not be an issue Because of what the cranks are going to do to my overall body position So I said to Neil over the phone I said can you explain that to everyone for me and he kindly said yes So I barged into his house two days ago and he's got some pretty interesting Insights to share about this crank length change and the positive impact It's going to have to my position making the chimney cut a non-issue But before he talks about that I got Neil to briefly discuss the methods of moving the seat up when you change crank lengths Because there are a ton of questions about that in the last video. So let's get into it Basically the easiest way to explain it is is as the crank gets shorter Your leg has to reach less far to the bottom of the stroke So it's logical that the seat height would need to go up to maintain the same kind of Angle if you like at the bottom of the stroke and and I don't measure seat heights Or or dictate seat heights by based upon knee angles at all I set seat height based upon control of the leg extension But in general it will need to go up now There's this occasionally There's a really a really rare possibility that there'll be an exception to that rule And the seat height won't need to change if the the person's a really extreme heel dropper Or the crank length change makes some other major sort of Change to their to their to their motion of their leg But in general the seat will need to go up somewhere between five and 10 millimeters So the variation depends a lot upon what the person does when you raise Sorry when you shorten their cranks Basically the seat needs to go up by an amount that varies sometimes between five and and and 10 millimeters It seems to be about the right range, but sometimes up to 15 millimeters If the person's having a lot of trouble controlling the bottom of the stroke with like a 172.5 crank And you shorten the crank by you know by seven and a half millimeters And suddenly their leg starts to gain proper control of the bottom of the stroke Sometimes that that that will really raise this that they're necessary Raise of their seat height will go up a lot and you might need to raise their seat up to 15 millimeters It all depends upon the person and I haven't seen you writing this thing yet But I said yeah, I'll just raise it five millimeters to start with But you know until I can see you on the on the bike and check out what's happening with your leg extension And so the the idea is that your seat height needs to go up But then that person if their crank length was limiting Their pelvic rotation forwards By shortening the cranks you will increase the ability of that person to to anteriorly rotate their pelvis to to roll their pelvis forward And as you do that if you imagine your hip is a ball and socket joint operating like this It's going to go forward and down right as you roll your pelvis forward. So that Then brings the center of rotation of the hip Forward and down closer to the cranks again So that can that can be where the the seat height changes more than the crank length changed So as as the hip goes down and forward the hip the hips rotational center gets closer to the crank And so you may need to go up and back More than the seven and a half millimeters And it depends basically upon how much that individual rider Makes that change And if they do it a lot you might need to go up by 15 millimeters, you know And and so i'm hoping that in your case your hip impingement, which is reasonably severe if it becomes Significantly unloaded as a part, you know on the shorter cranks your hips are going to roll forward and down More so you you will need to raise and possibly move the seat back a little bit That that varies a lot depending upon The rotational angle of their pelvis How much that affects their hamstring engagement at the bottom of the stroke? Some people as you roll their pelvis forward their their hamstrings go from being completely useless and doing nothing To suddenly engaging across the bottom of the stroke a lot more and then they will they will need to move the seat back Or or sometimes forward And they may bear their weight differently on the saddle as well And as they roll their pelvis forward The the sit bones the isiopubic ramai they they curve in three dimensions And so as you roll your pelvis forward you might sit deeper into the saddle or more shallowly up on the saddle Which can change the effective seat hiding as well So our hope with you is that because you you previously had this very rounded C-shaped curve in your back due to due to a posterior rotation of your pelvis um on the on the seat My hope is that if you start rolling your pelvis further forward here Your effective torso length that the you know if we took an imaginary dimension between your hip and your shoulder It's actually going to get longer because your spine will straighten out So you you'll go from being hunched to hopefully more extended And that can that can be a significant significant change And so the front end will need to quite likely go out And possibly down or just down or just out It could be it depends upon the person as to how much their spinal posture changes But steve hogg uses this um reference on his website, and I like it as well He calls it effective torso length because obviously we're not really changing the length of your torso But we're changing the shape of it, which effectively changes the length of your torso You know if you look at me from the side with a hunched body my head goes down You know 30 millimeters if I straighten my spine out it goes up 30 millimeters So your effective torso length on the bike can change quite significantly with a crank length change up or down If the hip impingement Is causing an issue like like it was for you So my hope is that you'll roll further forward here Your front end will go out and down and we may end up having to run a 10 mil longer stem And having to drop it 10 to 15 millimeters as well as raising the seat So this is where the Yeah, it may need to drop down to be you know the the holy grail of road cycling the slam stem right And you know that that would be nice because of the aerodynamic advantages that it that it brings to to you as a as a racing cyclist But we're not doing it to make you more aerodynamic We're doing it to make you more comfortable because if if you roll your pelvis further forward And your torso gets longer suddenly those bars are going to be too high and too close to you And you're going to prop yourself on them and bear more weight on your hands Like we talked about in the in the balance section of that of the video that we did before And so you may find that you get some unintended consequences of shortening the cranks like you start getting Numb hands because you're bearing so much weight on your hands And suddenly you need to drop down the bars and lengthen them out to get to get the weight to come back off your hands So it changes the whole kind of effective balance of the rider by changing their pelvic rotation And that I should qualify that that only really occurs if there's hip impingement In play quite quite probably. Yeah. Yeah, I teach for serious cyclists who've got good bike handling skills I'll often tell them to sort of ride with their forearms on the bar for a while Just just for five minutes two or three times a ride to condition that anterior rotation because it kind of forces them into that forward rotation of the pelvis And also to ride on on what I call the the joystick position Which is to hold the hood there and drop there drop their elbow down horizontal You know, they're forearm horizontal to the ground and that again will start to condition that anterior rotation of the pelvis And if you've spent a lifetime riding your bike with a with a posteriorly rotated pelvis And we suddenly give you the ability to to not do that It does still take a bit of time to change that that ingrained motor technique, you know that ingrained patterning So um You'll need to think about it briefly, but it will very soon very soon become normal to you And how did you go noticing the cadence increase that usually comes with it? So what i'm going to do next week is share my initial first impressions on the crank length change It's too early for a comprehensive review yet But certainly there are some things that I have noted that I think it's worthwhile sharing at this point in time Some things that are positive and also some things that i'm a little bit concerned about Now I wanted to quickly mention that the chimney that I talked about last week in the video I focused very much on aesthetics and I neglected A very important safety item Which i'm not going to reinvent the wheel There's a very good video that ray all did from lucid technique Which i'll link to below if you want to check that out and i'll leave you with some scenes now of my wife On Zwift doing some training. I'll catch you all in the next video It's going on boy What are you doing Alice? Baking a cake for you guys. Yeah. Yeah, you're doing new training sessions on Zwift. How's that going? It's good. I like it. I mean i'm watching it, but what are you doing? I'm watching Netflix On your phone That's pretty cool. Yeah, what show are you watching? I'll let you get back to it. Thank you