We will continue the discussion on finite element model updating, in the previous lecture we derived sensitivity of natural frequencies, mode shapes and frequency response functions, we considered situations where natural frequencies free vibration was done for damped systems and as well as undamped systems. So we postulated basically two states for the system, in the context in which we discussed this in the previous lecture there was one state one which is the postulated model for the structural behavior on which measurements have been done, the second state is the model on which experimental studies have been done. Now our objective is to arrive at a finite element model for the system on which measurements have been made, there is an alternative perspective which is related to this approach, this is related to problems of damage detection, it is closely related to problem of finite element model updating, here also we postulate two states, one is system in its healthy state, let us call it an undamaged state, then system which is postulated to be in a damaged state, so the overall damage here could mean changes in stiffness, mass or damping characteristics in the context of problems that we are dealing with, and our objective would be to you know analyze the measurements carried out on the damaged system and infer if the structure is indeed damaged, if yes where is the damage and what is the quantum of damage and more complicated questions like what is the residual life left and things like that. Now so in the discussion to follow we will try to adopt these two alternative perspectives, so it could be that we are dealing with a system on which the objective is not to do damage detection but to simply identify the parameters of the model, so the postulated finite element model is the first state of the system, and our objective as I already said to derive the finite element model for the structure in the way it currently exist. Now so we have FE model 1 and measurements made on existing structure which is model 2 and the updating procedure leads to the FE model for system in state 2, so depending on the based on the various sensitivity information that we studied we got the updating equation in this form, we have certain changes in response characteristics on the system that we call as delta gamma, this is to be determined experimentally, part of it is measured experimentally and part of it is postulated through the baseline, the finite element model for the structure and the difference is calculated, this S matrix is the matrix of sensitivity, you know sensitivity matrix, this has to be determined from the postulated finite element model, these delta is the vector of unknown updating parameters, so this typically constitutes a set of over-determined equation, this is a formulation based on first order sensitivity method and we obtain delta typically by using pseudo inverse of S into delta gamma, we can refine this procedure as I discussed in the previous lecture by using Tikonov's regularization strategies, so the task to be done is experimental determination of delta gamma and analytical determination of elements of S matrix that is sensitivity analysis, then solution of the over-determined set of equations either by iteration pseudo inverse singular value decomposition, Tikonov regularization, etc. Now this problem is notably different from the forward problem of design sensitivity in which we impart certain changes to system parameters in the parameters of the system and we would like to know what would be the change in response characteristic, that is the objective of a forward sensitivity problem, but we are using the idea of the sensitivity analysis in the inverse way, that means we have seen that certain changes have taken place in the system response characteristics and we don't know what are the changes in system parameter that have caused those changes and we use the same equation to find delta instead of delta gamma as in forward response design sensitivity analysis. Now there are many research papers on this subject of finite element model updating, there are some few useful references, this is a book by Friswell and Motorshed which is entirely dedicated to problem of finite element model updating and the book on model testing by Evans has a chapter on topics related to finite element model updating and in this collected papers edited by Maya and Silva there is a chapter again on finite element model updating. In the previous lecture we discussed briefly the quantity known as complex mode indicator function, so the mode indicator function are often real valued frequency dependent functions that show local maxima or minima at the system natural frequencies corresponding to the real normal modes, they can be used to identify repeated eigenvalues. So if H of omega is a N cross P measured frequency response functions we can do a singular value decomposition of this rectangular matrix and the matrix of singular values, you know we define what is known as complex mode indicator function in terms of singular values of this matrix and this is the definition. Now we discussed some of properties of CMIF and we illustrated this with respect to a 7 degree freedom system in which natural frequencies were repeating, but I would like to augment that discussion with 1 or 2 illustrations, so that's why this topic we have returned to again. The peaks in the largest CMIF indicate the location of natural frequencies, double or multiple natural frequencies are indicated by simultaneously large values of 2 or more CMIF values, associated with omega equal to omega R that is the Rth natural frequency, the left singular vector indicates the mode shape for that mode, and the right singular vector represent the approximate force pattern needed to generate response on that mode only, so this is something that is a property of singular value decomposition of the FRF matrix. Now we have considered few examples, I want to illustrate this with one more example, suppose we again consider a 7 degree freedom system, this is a stiffness matrix, this is a mass matrix and we use this as a damping ratios and C matrix can be constructed based on you know these modal damping ratios and that is given here. Now the question that I am trying to answer here is as you saw here we are talking about frequency response function which is not necessarily square, if you measure all frequency in a discrete multi degree freedom model the frequency response function matrix will be DOF by DOF, but often we will not measure all the elements of the frequency response function, so the point that I am trying to discuss now is what happens if we have a rectangular frequency response function matrix, so for this system if we have the full 7 by 7 of FRF matrix these are the CMIFs and the blue one is the largest you know the singular value and we see that there are 7 peaks corresponding to frequencies at which there are system natural frequencies, so this is fair enough, this is clear. Now if I now take FRFs to be 4 by 3 that means I will consider 4 columns, 4 rows and 3 columns and assume that only this is available, then we get that we are getting only 5 peaks, so CMIF from partial measurements may not provide all the natural frequencies, and this is another one where I have taken rows 2 to 5 and columns 1, 2, 3. Now in some other combination we see that all frequencies are captured, so this is something that one should be alert to when using CMIF with partially measured FRF matrix, on the other hand the receptance functions themselves may not capture all the natural frequencies, there are 7 frequencies from a mere visual inspection of this we will not be able to see the 3 more natural frequencies, but if I look at other elements of FRF matrix we will be able to see certain other frequencies which is not, is a mildly perceptible here it becomes more pronounced here, and with other by reducing damping I am trying to show that there is another peak here between 160 to 165 this is a zoom of H12 of omega amplitude of H12 of omega that is not perceptible here, but these kind of peaks are clearly seen if you look at CMIF plots, so a few comments can be made CMIF with full FRF provides clear indication of all the modes, CMIF with partial FRF measurement could miss some of the modes, elements of FRF matrix may fail to indicate at least visually the existence of resonant frequencies, the modal extraction algorithms may be able to pick them up although you may not see it in the FRF plots, the CMIF frequencies and MDO frequencies need not match, so this also is something to do with variation of mode shapes at the drive and measurement points because of which there will be again fluctuations in the FRF plots, therefore there would be differences, okay. Now we move on to a brief review of another question, how do we quantify the proximity of analytical model and experimental model, that is the question we are asking is how closely do the analytical and experiment models agree, so how do we quantify this? Now there are few metrics for this, this is what I will quickly review, there is lot of literature on this but I just want to give a flavor of what the issues are, now the preliminary step would be the matching of the size of, matching the size of the experimental and analytical model, this issue I have already mentioned in an experimental work the number of sensors you have may not match the number of degrees of freedom in a finite element model, often the number of degrees of freedom in a finite element model far exceeds the number of degrees of freedom in an experimental work, simply because some of the interior nodes will not be accessible, certain responses may not be easily measurable like rotation and so on and so forth. Now we can use either model expansion, that means the experimental model is expanded to match the degrees of freedom of the analytical model or a model reduction on the analytical model to match the degrees of freedom included in the experimental work, so for this typically we use SCREP, this I have discussed earlier in the context of model reduction in one of the previous lectures. Now the basis for setting up measures for comparing analytical and experimental models is again something to do with orthogonality relation, so we define Phi to be mass normalized model matrix, Phi transpose M Phi is I and Phi transpose K Phi is diagonal matrix of squares of system natural frequencies, we use the subscript A to indicate analytical model and subscript X for experimental model, and Phi we use for mass normalized mode shapes and Psi is arbitrarily normalized mode shape, so this is the nomenclature subscript here refers to analytical model subscript X refers to the experimental model, so mass normalized normal modes will satisfy this condition, if it is not mass normalized this matrix will still be diagonal but the elements will be, you know the generalized masses will not be identity equal to 1. Now there is a quantity known as normalized cross orthogonality denoted as NCO, it's a quantity that is computed between two modes, suppose from the experiment you pick the ith mode and from the analysis you pick the jth mode, we define this quantity, this is Psi X transpose MA Psi A you know analytical, MA is analytical mass matrix which will be available to you. Now what is the issue here, the issue is when I experimentally extract certain modes, natural frequencies and mode shapes and I computationally predict them how do you order, how do you pair which mode in analysis corresponds to which mode in experiment, so we call them as correlated model pairs, so it is very important that we establish the correct pairing between the modes predicted from analysis and experimentally measured modes, it is quite conceivable that in an experimental model we may miss some of the modes, so suppose if you measure 5 modes experimentally and your analytical model has 25 modes, which 5 of the experimental modes correspond to which of these 25 modes in a computational model is not a question that can easily be answered, so we define the quantities like NCO to able to answer that, so this is as you see it is defined in terms of arbitrarily normalized real normal modes, and you can see that if Psi X is Psi A that means the experimental and analytical mode shapes match this quantity will be 1, if they are very much different that quantity will approach 0, values closer to 1 indicating the closeness of analytical and experimental mode shapes, now if M is a number of experimental modes and N is a number of analytical modes NCO would be M cross N matrix, okay, then those elements of this M cross N matrix which are close to 1 provide the correct strategy to identify the correlated model pairs, so first mode in experiment will be poorly the NCO between first mode in experiment and say third mode in analysis will be close to 0 if these modes are correctly nomenclature is correctly assigned to them, you follow, if you are dealing with say third mode in experiment and third mode in analysis NCO will come pretty close to 1, so this helps in identifying the correlated model pairs, now if Psi X is alpha times Psi A NCO will be an identity matrix, that means if mode shapes are proportional, okay, this NCO will be I, now this as you see requires the mass matrix of the analytical model, we can define another quantity known as modal assurance criterion, here we don't use any structural matrices, again this is defined between ith experimental mode shape and jth analytical mode shape, this is a MAC of between these two modes is given by this quantity, these are all intuitively defined quantities, there is no mathematical basis for arriving at these quantities, now you observe this here we are allowing for complex valued mode shapes, that means influence of damping in determining mode shapes is included, as I already pointed out in an experiment you will be always be measuring damped normal modes, now this can be used in conjunction with coordinate incomplete mode shapes, for example the analytical mode shape can simply be partitioned instead of doing a modal reduction or expansion, and MAC also lies between 0 and 1 and values closer to 1 indicating the closeness of analytical and experimental mode shapes, again MAC would be a M cross N matrix, if M is the number of experimental modes and N is the number of analytical modes, this matrix facilitates again establishing the correlated modal pairs. Now if Phi X is alpha Phi A in this case MAC would not be an identity matrix, because this is not, there is no mass matrix here, you recall that the mode shapes the way we are deriving by solving the eigenvalue problem associated with K and M are orthogonal with respect to mass and stiffness matrix, so Phi transpose Phi is not in a diagonal matrix, it is Phi transpose M Phi is a diagonal matrix, so this won't be equal to an identity matrix. Now there is another quantity known as normalized modal difference and modal scale factor, so again this is defined with respect to two mode shapes, this is given by this quantity, Phi X of I minus gamma Phi A of J, this is L2 norm divided by L2 norm of gamma Phi A J, and this gamma itself is, this is modal scale factor between the two modes and this itself is defined in this manner. We can show that this normalized modal difference is related to MAC through this equation, I leave this as an exercise, again here no FE matrices are used, but the utility of this notion is that mode shape at each DOF for example is erroneous by 10% then this NMD will become point on, so it gives you directly a measure of error in a more direct way. Now this MAC and NCO compare two mode shapes and give a measure of which mode has to be paired with, from experiments with which mode in analysis, but they do not tell where is the location of difference between analytical model and the experimental model, so the spatial information is not included there, so to do that we ask the question how to locate spatial regions where the differences between analytical and experimental modes shapes are the most pronounced, okay, that is where you may like to update the modal parameter, that is a hope, but there are catch, there is a catch in that which we must understand carefully, so with that in mind we introduce what is known as coordinate modal assurance criterion, here let NCMP be the number of correlated modal pairs which has been established by some criteria, then Phi IJ I define as value of mode shape at the ith coordinate in the jth mode, so I is the coordinate, J is the mode count, so the coordinate modal assurance criteria for the ith coordinate, okay it is defined with respect to a coordinate not with respect to a mode, and all the modes are summed here, this metric is obtained by summing over the mode shapes at that value of the spatial coordinate or the degree of freedom, that is defined in terms of the analytical mode shape and the experimental mode shape through this relation as shown here. This Comac also lies between 0 and 1, it has no physical basis it is intuitive, it can be displayed as a contour plot over the domain of the structure, that means you can plot the structure and you can have color coding for different values of Comac or draw contours, and where Comac is close to 1 we suspect that the agreement, disagreement between the analytical model and experimental model is most pronounced, regions of low Comac represent the regions where the consequences of difference between analytical and experimental modes are felt most pronounced sadly, these need not be the region where you have actually made errors, there are errors in parameter values, it is only the place where the consequence of the difference is felt most pronounced sadly, so the for example in a cantilever beam if you have made an error in modeling boundary condition at the fixed end, then the effect of that error will be most pronounced at the free end of the cantilever, you might have done modeling at the free end appropriately, but the error done elsewhere the consequence of that will be felt somewhere else, so that precaution you have to take, but typically we tend to use Comac to identify those spatial regions where we want to correct the parameters, but while doing so that is a hope that Comac indicates that, but there is it is not guaranteed to do that, but it gives you a kind of a hint that there could be something wrong with properties in those regions where Comac is low, okay, so that is how it is, this is done. Now there are few other metrics I will just for sake of completeness briefly mention them, suppose you have determined mode shapes, suppose Jth mode shape has been determined and Jth natural frequency has been determined, and if I now plug it back into the analytical model K into phi minus omega square MA into phi must be 0, that is the eigenvalue problem that you need to solve, but if there is an error there will be a residual force, that again tells you where is the trouble, okay, so if these two are equal that is phi XJ is equal to phi AJ, it is automatically satisfied because KA and MER analytical matrices and phi AJ is analytically determined mode shape that has to satisfy the eigenvalue statement of the eigenvalue problem. Now to implement this idea you require coordinate complete measurements, that means this phi XJ should be, the size of phi XJ should be equal to the degree of freedom in your analytical model, so you may have to expand the experimentally measured mode shapes using a suitable strategy, so the answers should be affected by the strategy that you employ, so the aspiration is it would help in locating regions at which residual errors are high, the errors could be in stiffness or mass that issue doesn't get resolved here. Now this is the metrics that I have talked about are in terms of natural frequencies and mode shape they can be damped or undamped, we can also define similar quantities as frequency response functions, analytically we can predict the frequency response function and experimentally also we can measure the frequency response function. Now on the lines of MAC and COMAC one can define frequency domain assurance criteria called FDAQ and frequency response assurance criteria called FRAC, wherein one uses measured and analytical FRFs instead of mode shapes, so in the definition of MAC and COMAC instead of using mode shapes you use the measured frequency response functions, so you get FDAQ and FRAC, so they again serve the same purpose as MAC and COMAC by and large and it doesn't require extraction of mode shapes and natural frequency. Now this is just a brief overview of the model correlation methods, again you need to go back to the references that I provided especially the book by Evans to you know completely understand what the issues are, so here in these two lectures on financial model updating I am trying to provide a flavor of how to approach these problems and what are the issues and some simple illustrations. So the illustrative examples that I am going to present now are with respect to a class of hypothetical multidegree freedom systems, and we have done some studies on simple beams and building frame models experimentally, and the presentation of some of these results will now follow. In the first study we want to you know apply the inverse eigen sensitivity analysis for undamped systems, so the objective of this illustration is evaluation of mass and stiffness parameters, and study the effect of including cross orthogonality relations in deriving the updating equations, so this is quick recall of the governing equation, this is a equilibrium equation, this is a assumed solution, this eigenvalue problem to be solved, these are the natural frequencies and mode shapes, these are the orthogonality relations. Now in the previous lecture we have derived the equations for sensitivity of eigenvalues and eigenvectors, and based on that we formed the updating equation and we got the solution and as delta is S pseudo inverse delta gamma, now we will apply this on a 5 degree freedom system as shown here, now these are the masses, these are the stiffnesses, and for this system these are the natural frequencies, and this is mass normalized modal matrix. We call this as a baseline model, and I am going to alter some of the properties of mass and spring and call it as system in damaged state, so if it is, if we are not talking about damaged and undamaged systems it could be a postulated finite element model and the model, unknown model from which we have taken measurements, in the illustration that I am presenting in this part the experimental results are synthetically generated, there is no true experiment but numerically we are doing an experiment, the idea here is to see how the updating equations can be implemented and what if any are the pitfalls in using that, this is an exercise that is worth doing before you actually start working with experimentally observed structural properties, so what I will do is I will use the language of damaged and undamaged systems, so MI in damaged state is taken as alpha into MI in undamaged state, the MI, KI and CI are these discrete elements shown in this figure, so we introduce alpha I, beta I and gamma I, and the updating parameters consequently are these non-dimensional numbers alpha I, beta I and CI, if the structure suffers no damage all these quantities will be equal to 1. Now we will adopt two methods, in first method we will not include cross orthogonality relations, in the second method we will include the cross orthogonality relations and see if there is any advantage in doing including cross orthogonality relation. Now there are two damage scenarios with various levels of severity, the notes for this presentation will be available with you, you can study this in greater detail, I will now present the main features of the result without getting into the base all the intricate details. Method one where cross orthogonality is not included predicts the five masses in acceptable manner and stiffness parameters also in an acceptable manner, for the method one that is case one, for case two what happens the method doesn't perform acceptably, see for example the on X axis in all these plots is the iteration, the global iteration step that I mentioned in the previous lecture, you start with initial guess and improve upon that successively through an iterative process, here you can see the algorithm is not converging whereas you can see here all the system parameters have converged and have become constant after about 10 iterations, whereas here they seem to be diverging here. So method one does not perform for case two of the so called damage scenario, now you include now the cross orthogonality relations we see that case one the method works, case two the method indeed works, so this is an example where the objective is to illustrate that by including the cross orthogonality information in deriving sensitivity information we are able to achieve better solutions in terms of determining finite element model updating parameters, this study was for undamped system, in the next study we are considering the same approach inverse against intuitive analysis but now for damped systems, so the objective here is to evaluate mass stiffness and as well as damper properties, then again study the effect of cross orthogonality and how to carry out analysis with complex modes that is the objective. So we rewrite the equation of motion in this form AY dot plus BY equal to F of T and we get the mode shape modal matrix in this form and eigenvalues are N pair of complex conjugates and these are the mode shape both are complex valued and the orthogonality relations are as here. So this we have done the sensitivity analysis in the previous lecture and this is the using first-order sensitivity method this is the updating equation that we need to solve, so these are complex valued and we separate the real and imaginary parts and arrive at the appropriate simplified updating equations. Now in the numerical illustration I again consider a 5 degree freedom system, these are the masses and stiffnesses and these are the 5 damper elements and undamped natural frequencies are this and this is the modal matrix and I am applying this method now on a proportionally damped system, so we can use complex normal mode analysis on proportionally damped systems also, it doesn't prevent us from doing that. So these are the natural frequencies and these are the damped natural frequencies, clearly you will see that there is a relationship between these damping ratios and the quantity shown here. Now this is the modal matrix you can understand this right now it has this structure that phi lambda, phi star, lambda star, phi and phi star, so you can see that such a structure exist here. Now the problem is we will introduce two dimensional scenarios, we will simultaneously change the mass properties, damper properties and stiffness properties and there are two such scenarios that we are adopting and the objective of this study is to see how the first order sensitivity method performs by including cross orthogonality relations and by excluding cross orthogonality relations. So for method one the first case the seems to perform reasonably well, the second case shows certain instabilities it's not satisfactory, there are certain perturbations and so on and so forth. So we use method two the solutions are much well behaved, all the stiffness parameters, damping mass and damping parameters for both the cases show stable behavior as iterations proceed, okay, here again we reach the similar conclusion that including cross orthogonality relations is helpful in getting better solutions. Now we also seen that we can perform inverse sensitivity analysis using frequency response functions themselves, so we need not have to extract natural frequencies in mode shapes and do this analysis, we can directly do the analysis with the measured frequency response functions, this we have derived in the previous lecture so we will not repeat that, in a first order frequency response based sensitivity analysis this is the finite element model updating equation and we will illustrate that now, we will take a baseline model the damping matrices, matrix is simply taken to be diagonal and this is for again for illustration these are the undamped natural frequencies and this is a mass normalized model matrix. Now there is a damage scenario as depicted here and there is a, we are going to use first order sensitivity analysis and also we are going to illustrate the functioning of second order sensitivity analysis where we have increased the level of damage to see whether we gain any advantage in using second order methods. So for the method one and damage scenario one the first order sensitivity method seems to provide reasonably good answers, so this is fine and second order method also provides good results, both the methods seem to perform well here, now we talked about inverse sensitivity of singular values of FRF matrix, now here we will select an example where our objective is to do damage detection in systems with repeated or closely spaced modes and identify damage parameters in the systems, so we synthetically simulate a few situations where before damage the structure has distinct modes and because of damage the system will have repeated modes, this is an artificially simulated scenario but it is useful to see in these limits how the system performs and the solution strategy performs. So before damage the system has repeated modes and after damage also it continues to have repeated modes and third case is repeated modes before damage and distinct modes after damage, so how these methods perform? So in state one there are either there is a repeated natural frequencies or not in the state two the similar thing with different combinations are used, so we have gone through this definition of singular values and things like that I will not repeat that, so we consider this system now, so this again a 5 degree freedom system now it is supported and connected in a slightly different way, so we consider now the scenario where these are the properties and in undamped natural frequency you will see here that the third and fourth natural frequencies are repeating, okay. Now we will introduce a damage scenario and here if you see because of this damage scenario the third and fourth natural frequencies still repeat but at it they have a different value, okay, so this is in the damaged state this is undamaged state, so we have undamaged structure with repeated eigenvalues and damage structure with repeated eigenvalues, but the natural frequencies are having different values, okay, so this is how this artificially simulated, so we have in method one it is inverse CMIF sensitivity analysis, method two is inverse eigen sensitivity analysis of real modes without cross orthogonality and inverse eigen sensitivity analysis with cross orthogonality, so method one seems to perform reasonably well and method two there are these problems here it's not working nicely, method three cross orthogonality is included it seems to converge to give the convergent answers but we have to see whether that is right or wrong, so this is just an example so you can verify whether these results are acceptable. Now there is yet another method known as frequency response function method, this we have not discussed earlier so we can quickly review what this is, so here it is evaluation of mass and stiffness parameters with model reduction, now let's consider the equilibrium equation of the system in the damaged state and this is the equation, now XDs are the degrees of freedom whose size is equal to the XA which is analytical degrees of freedom but we have to do a model reduction so that degrees of freedom match and we use this transformation matrix either could be condensation or ACREP, we use ACREP and substituting that I get the reduced equation and this is the equation which we will use to predict the measured FRFs, so the FRF predicted from this will not match the measured FRF, so that by writing this equation at different frequencies we can adjust the parameters of these models and get adequate number of equations to solve that, so at a frequency one value of driving frequency if you arrange these terms we will get see UKVK and WK are this and I am putting it here I get these equations, and if I repeat this equation for a set of Q frequencies at which FRFs are measured I get this over determined set of equations and I get the updating equation in this form, so this is straight forward conceptually there is no problem but there is a model reduction step that is involved, so this we have applied on one of the examples I am just flashing the results I implore you to study this and maybe verify the results shown here, so this is the before updating, this is before damage detection and updating you see there are differences and after the updating process is completed you see that the matching is perfect thereby indicating success of the updating procedure. Now we can summarize what we saw, so what we have seen is methods based on sensitivity of undamped Eigen solution can detect changes that occur in stiffness and mass properties, the other inverse methods are all capable of detecting changes not only in mass stiffness but also in damping characteristics potentially they are capable, inclusion of sensitivity information with cross-orthogonality relations helps in identifying systems which are higher levels of damages, inverse CMI of sensitivity method successfully identifies damages in system with repeated modes while inverse Eigen sensitivity method is found to be unsuccessful in such cases. The numerical investigations have revealed that the method based on inverse Eigen sensitivity that includes complex nature of the Eigen solutions and information cross-orthogonality seems to perform most satisfactory except in situations where you have repeated natural frequencies and such exceptional situations. Now all these illustrations were with respect to synthetic examples where measurement was artificially simulated on a obviously a overly simplified idealized system. Now finite element model updating is actually meant for studies that are actually performed in laboratory on existing structures, so to explore how the methods that we have discussed perform in a laboratory condition we have considered three problems, one is a cantilever beam with inhomogeneous distribution of mass properties, other one is a free-free beam with inhomogeneous mass and stiffness properties, and the three storage here building model with inhomogeneous mass and stiffness properties. So in the cantilever beam, first example is a cantilever beam, this is shown here you can see here this is a cantilever beam, and what you see here as white boxes here these are the accelerometers, and the person here is hitting this beam with a instrumented impulse hammer, and this is being done to measure the frequency response function, and if you see carefully here the measured frequency response function is displayed here, so this is a complete experimental setup which involves instrumented hammer and sensors and a computerized data processing system. So now what we have done is there is a mass M whose position can be varied, and we assume that in the structure when it is in healthy state this mass is placed here, and upon occurrence of damage this mass is removed, so we get system in two states and on both the beams in both these states we perform this experiment and try to identify the properties, and the question is do we really detect whether where was the mass before it was removed, it is a self-validating exercise in the sense I know where the mass was, so it would help us to understand how the method works. There is one complication here which is invariably present in experimental work involving fixed and pinned boundary conditions, see if you see carefully see here this beam is clamped to a rigid block here with two bolts, it is not clear whether this rendement is adequate enough to deem this support conditions as being fixed, in a static sense may be yes but in high frequency vibration problems we are not sure whether that end is truly fixed or not, so what we do is in identification problem we add a rotary spring here to indicate that there is a partial fixity condition that we need to be aware of. The second experiment is done on a free-free beam, there is no problem of you know supporting the beam but it is suspended through flexible wires ropes as shown here, and this person is again hitting this beam with impulse hammer to measure the frequency response functions, so there are various configurations of this beam that have been created by adding certain stiffener, so if you see carefully here there is a stiffener that is added and there is also a mass that can be moved, so there are four configurations here as shown here, this is ABC or the damage configurations, and this is the undamaged configuration that is where all these stiffeners and mass elements are shown as here. Now the third example is on a three-storey shear building frame, here there are system in two states are displayed here, you can see here in this state there is a mass that is placed on this slab and also there is a stiffener that is added in the ground floor, so by removing this we create this state, so we measure frequency response functions on these two systems and by comparing the differences in natural frequencies, mode shapes, FRFs, etc., we would like to identify the fact that this structure has been obtained from this structure by removing these known elements. So configuration 3, I mean we can define several configurations, in configuration 1 the stiffener and mass are present, in configuration 2 all of them are removed, in configuration 3 we can remove only one of these, either the mass or the stiffener, so we can create by different configuration, a different combination of removal of these elements for different configurations, so that is outlined here, and this is experimental system, this is a extra mass that I have mentioned, and this is instrument at hammer that will be used to measure the frequency response functions. So we have performed the studies that I have mentioned, so I will leave these details in the notes, I'll leave it as a reading exercise for you to go through these three examples and see what conclusions have been reached. These are the experimentally measured frequency response functions, this is an amplitude plot and these are the phase plots at different locations, and we have, as I said I am not going to discuss all the details, here we have computed the modal assurance criterion to pair the experimental and experimental determined normal modes and analytically determined normal modes that is also provided the details are provided here, and these are some of the results of COMAC and detection results of damage detection. So here I am showing the experimentally determined normal mode and analytically predicted normal mode, the red one indicates the experimentally determined normal mode, so these results as I said I am not going to discuss this is for as a reading exercise for you to understand what these are, so this is for the case of the shear building model, this is how the two FRFs before you know we do damage detection and predict the behavior appear, and after the damage detection has been performed this is how it is reconciled. So the summary of the findings of these investigation is that the performance of damage detection algorithms when applied to synthetic data was found to be generally very satisfactory, that is to be expected, it validates the procedures developed and the way they have been coded and the method has been implemented all that is validated, this is an essential first step if you are going to do any of the, implement any of these methods in practice, first you can try out all the algorithms and coding with respect to a synthetic example, where you know, what is the change, where is the change, etc., when applied to experimental data we will not know, what is the actual true parameters would be unknown, so whatever the method tells we have to you know accept in some sense. Now since we know in these examples what was the change that we have made to the structure, because we added a stiffener, removed a stiffener, added a mass, removed a mass, etc., etc., we have some idea of what is the changes that we are making, so based on this it is concluded that about 0.0 to 10.7% is a accuracy for beams using inverse eigen sensitivity method, and this is shoots up for FRF based method, and inverse eigen sensitivity for the building frame, this is the error that we observed, and this is based on FRFs, so there is no definite recommendation that we can make which method works, so it seems to, it is fair enough to say that the errors that we encounter depends on the situation, all the methods can potentially lead to equally good results, if applied correctly. Before I close this discussion I would like to briefly touch upon a few other finite element topulating methods, we have basically discussed inverse eigen sensitivity and FRF based, frequency response function based methods, but there are other strategies also, so I will quickly review them, the first one is known as direct matrix method, so what we, here what we do is, for example MX is a mass matrix for the experimental model which is not known, we assume that we propose that the elements of MX can be determined by minimizing a metric delta M as defined here, so MA is analytical mass matrix, MX minus MA can be, you know is defined, this is an error, and this is not known, MX is not known, so elements of MX are the variables of optimization, similar statement can be made for stiffness matrix also, so now if you are focusing on finding elements of MX which minimize this delta M, we need to put some constraints, for example mass matrix has to be symmetric, stiffness matrix has to be symmetric, and these orthogonality relations must be satisfied, where phi X and lambda X are experimentally determined, now this is one of the earliest methods of finite element model updating that was developed, but the problem here is the physical connectivity of structural model may not be honored, if you are simply changing elements of mass matrix you may get a non-zero entry in the mass matrix which is not supported by the actual connectivity that you see in the existing structure, so that is not imposed as a separate constraint, and also the changes that we observe that MU minus MD and KU minus KD are not guaranteed to represent physically meaningful quantities, so this is not quite satisfactory, Remedy to some extent to this can be formulated as follows, so what we can do is if there are N elements that is N ELM number of finite elements in your analytical model, we can assume that each of these analytical mass matrix of the analytical finite element model need to be corrected by a factor AI to obtain the corresponding mass matrix in an experimental model, which is not known actually, so AIs are scalar factors associated with each element, similarly this is BI with stiffness matrix, now what we can do is we can formulate the experimental mass and stiffness matrix in terms of these unknown AIs and BIs, and we would have measured the normal mode, say phi X is measured, so we can substitute them into this equation and see we will get a set of equation from the two orthogonality relations and they can be recast in this form, that means we are actually constraining these AIs and BI's by the orthogonality relations of the normal modes, and the relationship between natural frequency and mass matrix, sorry stiffness and mass matrix and the Eigen solution, so this leads to bisuitable manipulation a set of over determined equations and this is what you have to solve by doing maybe pseudo-inversing with regularization and so on and so forth. In a more direct approach, suppose you have measured Q number of natural frequencies and R number of mode shapes, or S number of mode shapes at R number of spatial coordinates, so we can define a metric of differences, so this is the sum of squares of observed differences between the experimental and analytical natural frequencies, this is the sum of squares of difference observed difference between experimental model and analytical model summed over all mode, summed over all the space, so J is a positive quantity because we all squared this. Now this J will be function of the unknown system parameters, so what we can do is we can optimize J, we find P which minimizes J, so and suitable constraints can also be imposed in solving these equations, those constraints reflecting the basic dynamical characteristics of the structure. So the other method that I briefly mentioned here the Bayesian filtering method I remarked in the previous lecture, this is one of the most powerful methods, this is based, this has roots in probabilistic methods, we apply Bayes theorem, we treat the mathematical model and the measurements as being imperfect, and we use probabilistic arguments and derive the posterior probability density function of system parameters conditioned on the measurements made, so in this approach all the system parameters are treated as random variables, so this can be applied to linear problems, nonlinear problems, and in fact the resulting equations can be solved using Monte Carlo simulations, but as I already said the scope of this method is outside the purview of this course, so we will not be elaborating further on this. So with this brief overview we will close our discussion on finite element model updating, and in the next remaining part of this course we will look at some problems of nonlinear structural dynamics problems, so how to deal with nonlinearity especially in the framework of finite element formulations, what really happens, can we still formulate element matrices, can we still assemble, how do we solve the equations, all these questions can be posed and we will find suitable answers to those questions, so that we will take up in the next lecture, we will close this lecture at this point.


The Minister of State, Petroleum Resources, Timipri Silver, has begged Nigerians to exercise patience over the touches challenge of petrol scarcity, assuring that efforts were on to end the problem and avoid a repeat of it. Silver's apology was contained in a statement signed by Haritius Egoa, his senior advisor media and communications. The minister reckoned that Nigerians in recent weeks have grippled with fuel scarcity, not because of the absence of supply of product, but due to inspection failure and regulatory negligence which allowed adulterated products into the country. He added that while the development was regrettable, it is a time that calls for collective action to save a situation that was not foreseen. Meanwhile, fuel queues have substantially thinned out in various parts of the federal capital territory on Sunday. The development seems to be a confirmation of government's assurances that fuel queues would gradually disappear from this weekend when 3.2 billion litres of safe petrol imported is injected into the domestic market. A lot of fuel in stations that were heated out of stock for weeks were seen selling petrol to mattresses. However, it was observed that consumers preferred patronising outlets of major oil marketers for fear of not buying remnant of contaminated petrol.


Alright, hi. Thank you all for coming to Reversing Corruption Seagate Hard Drive Translators. Also known as the Naked Trail Data Recovery Project, because we've had some technical difficulties. I may speak really fast. I'm from New York. I'm sorry. Um, but thank you for all making through the maze and through the confusion to be here. I really appreciate it. My name is Allison. I'm one of several Allison Noctaborns. Uh, I'm a software developer by trade. I build high level blue, blue software. So this has nothing to do with what I do as a day job. I haven't really touched hardware since college where I did a bunch of robotics research and learned that cutting edge research is the easy part of getting published in IEEE publication. It left a little bit of a mark, about a dozen years worth of a mark. So that's really it for my hardware experience. I am not an expert in this. That's her. That's her job. So before you tell me I'm wrong. That's a slow transition. Look, dude, I'm Mr. Dead. Um, I named the project. It's a play on our last names. You know, uh, Born Naked is her last name in Dutch. Um, I've given Tux talks at HushCon, Teardown, and then Archreactor Hacker Space. Uh, I founded Revenant Data, which is a data recovery company in Oregon with Wireglitch who's somewhere around here. Um, I've co-founded a couple of Hacker Spaces, uh, the most recent one, like Pascal and PDX Hack Labs. I, I try to do, like, uh, workshops on reverse engineering and focusing on assembly and binary challenges. I don't have a formal education. I did not go to Carnegie Mellon University, like Allison. Everything that I know is self-taught, like minus, I, I did do Scott Moulton's day recovery training, the remote one. Um, I've never actually met him, which is kind of funny. Um, and I do roller derby and I like stress bake. I get stressed out and I bake stuff. That's my non-infosec hacking stuff. Alright. The original quest. Uh, we dubbed it the seemingly undamaged HDD in a really bad day. Uh, so what ends up happening is, this is out of order. Oh, that's fine. I get a lot of these cases in, translator corruption, which basically means I have zero access to user data or sometimes I'll have partial access. Uh, what, that, how that manifests is it'll boot, spin, it will get stuck or never, and by, you know, stuck and busy, it just means not recognized by the host. Uh, and then it never initializes. Another thing that can happen is it will boot, spin, click, not all clicks or clicks of death. Uh, and then it'll spin down. So when that happens, um, especially if there's clicking involved, and this is always the type of corruption that we're talking about occurs in Faro Seagates and Moose Seagates, which I know many of you may not actually be familiar with family names in hard drives, but you can ask me about that later. Um, so when the clicking happens, it usually is indicating the short points problem. Uh, which means that I have to short the read pins or short the read points in order to gain terminal access and then force a regeneration of the translator. Success.  Success, but it's actually never really that easy. Um, it can become an, like an extremely awful time sink for me. Uh, like in order to, I have to short these read pins at a very precise time and then I have to send a control Z to gain terminal access at a very precise time and it can suck up anywhere between an hour of my time to six hours. I'm not kidding. My, uh, partner and I would often have to like switch off because we'd get tired of like sitting there and listening for the heads to leave the platter and then like start mashing control Z. So, um, what makes this extremely frustrating is once I get terminal access, forcing the translator regeneration is actually very simple, but it's this timing issue that drives me nuts. Long story short. Um, I went to the, the hacker space and started complaining about this a lot. I really don't like this. Yeah, but anyway, um, Allison overheard me and I, I said, oh, I want to automate this. I would love to be able to bypass this like four to six hour timing issue of shorting these points and sending a terminal command so that I can just boom, get to the translator regenerate, regeneration. And she chimed in and she's like, oh, I can script that. Um, I think you said, hey, I need some code. It'll be really easy. I'm pretty sure that's what she said. I may have tried to sucker her into it knowing damn well because I'm not a coder. I don't like I, I'm very little programming skills. Um, so along the way, while we were doing research to try to solve this particular issue, we started realizing there's a bunch of other little like much simpler tools that we could fit or that we could build. And we started doing that. Um, cause the ultimate goal where we are currently focusing on translators and also just Seagates, even more specifically these two families. However, several of our tools do work on pretty much any Seagate. Uh, 7200 dot 12 and beyond. Um, any who, yeah, we started, uh, working on other types of tools that we can fix. We want to turn it into a full on data recovery suite that's open source. Reasons to care. I know a lot of people think that hard drives are obsolete or near obsolescent, but, um, there are some pretty important reasons to care. Number one, right to know, uh, there are standards for hard drives. A lot of you are probably familiar with ATA standards, the INCITST 13 committee, they issue those. Um, the problem is, is majority of these are vendor specific. The tool that we built, we use the, um, terminal debug interface that has zero standards. Everything's vendor specific. So if you don't know what you're doing, you either have to send it to a data recovery company. It's going to cost you an arm and a leg, most likely. Or you have to sync, honestly, probably months if not years of research into figuring out what commands can I use, what does this command mean? Um, these lack of standards make it really difficult to, like, implement quality assurance. And again, you're forced to rely on very costly data recovery tools or services. Um, right to repair, I whole hearted believe ownership is not a timeshare. Uh, it really bugs me that manufacturers want you to use their data recovery services only. Um, again, I think that ultimately drives down QA.  Um, they're manufactured with cobalt on the platters. Um, that can, it can cause neuro degenerative diseases. It's been linked to ALS recently. And I do have citations for all this. And then of course silica also neuro neuro degenerative diseases. That's you. Yep. This podium is freaking me out. All right. So why you know Linux? Yeah, yeah. And apparently I can't use Windows. So I suppose I don't have any stones for a, all right. So of the discs that cross Mr. Dead's desk for repair, about 50% of them are Seagates of that just over 8% display translator corruption. This doesn't sound like a ton, but um, scale is a funny thing. So Seagate started shipping drives in 80 1980. And about 20 years later, they announced that they shipped 250 million hard drives and they've kept shipping at about that rate. So let's do some bar math. Um, translators really first appeared with the F3 architecture, which was shipped in about 2008. And if we say there are 100 million drives active, then we are looking at north of 8 million cases of translator corruption. So the odds are good. If you have a couple drives or you run a data center, it's going to happen to you. And when it does, you'll find out that this can be really expensive. We did some social engineering and we found that the cost of a translator repair can run you somewhere between $703,000. And in general, repair on a hard drive can run you north of 8,000. So getting your bits back can be extremely expensive. And I'll know about you, but I don't necessarily have a spare 8k laying around. So that's the reason you should care beside the high minded stuff. I'm going to try to get through this because I know it's kind of boring, hard drive anatomy. Number one, filter, not very important. Number two, however, those are the platters aka the media aka spinning disks. And that's where your data is technically actually stored. What else to pay attention to six, the the heads, right, the read and write elements of the drive. They're going to fly over the platters, reading and writing data. They're parked on the head ramp there. But honestly, the second or, you know, third most important element is seven. That's the head stack connector. So that's connected via that number eight, the ribbon to the actuator arm. And then on the other side, that's touching the PCB. And just remember, I was talking about shorting read points. Well, I have to know where those are because that's what I'm doing. I'm preventing via this solution. Though the read like the the heads seeking the service area, it'll stop I'm tricking it into giving me a debug mode. So anatomy, hard disk geometry. I wanted to bring like I wanted you all to kind of have a little bit of background on this because you know, CHS, this is introducing you to CHS and LBA service area is that grayish blue ring around the outside if they're outer parked heads on a ramp. It could also be on the inner ring, which that would be inner parked heads so heads are parked on the platter. I can't issue this solution with those types of drives all the time. It's harder because I'm literally listening and also kind of feeling for when the heads are leaving the the ramp. To the sectors which is the S and CHS. That is physical location of data. It is the smallest unit of data on the hard drive. 512 bytes of user data, 520 if you factor in ECC error correction code. And is then divided up into cylinders which are concentrically expanding rings from the center of the drive. If you have multiple platters, that's going all through every platter. I highlighted it there. Number four, that is a cylinder. Those are all cylinders. Some people will dispute and call them tracks. That's a whole big debate. And then that cylinder again, that would be the C and CHS. The H is the heads. So cylinder head sector. And a zone. That includes like several cylinders. And it is just a way to like aid in translator. But we're not going to get into that. There you go. It's you. Hardware is icky. Let's talk about software instead. And guys in the back, feel free to come in and sit down. We got plenty of seats. So what's inside the firmware that we really care about? Well, one of the most important data structures is the translator. And the translator maps from logical block addresses to cylinder head sectors. Anytime you see the word logical in a name, it's a Q that we made this up. Logical block addresses are an abstraction. CHS represent a location in physical space on the disk. Your translator will map between one and the other and vice versa. Well, how does it do this? Well, let's like all good data structures or famous hackers. It has a posse. Its posse is made up of defect lists. Your translator is a great place to hide things if you don't want the host OS to know they're there. Not that I would encourage you to do this. But in theory, it wouldn't be half bad. So the lists that you use as a helper to construct the translator as a data structure are two types of lists. We have primary defect lists and ground defects lists. Primary defects are defined as defects that are found at factory or creation. You're either booked or you're not. Sectors are just kind of bad sometimes. We make them, no one makes a perfect disk. Ground defects are the ones that accumulate over time. Flakes of rust, cosmic rays, aliens staring at your hard drive, stuff just goes bad over time and we know it. So we tend to stick them in two lists. The ground defect list usually has a lot more space than the primary lists because it's the size of that tends to determine how long your drive will really stay viable. In Seagates we find a second form of primary lists called the non-resident ground defect list. This showed up in... It showed up after 7200.12. It doesn't exist in 7200.11. Ask her. The architecture came out in 2008, so around then. But ask me about that later. Anyway, go, go, go. All right. These two lists are created by something in Seagate Lingo called a self-scan test. As far as I can tell it's a QA test they run. And when working with these lists they appear to largely function as glorified arrays but I haven't been able to fully prove that's what they really are. They just seem to act a lot like it. So, all right, let's talk about some lists. No, no, no, not lists. This is physical to logical. I was getting there. Oh. Wait, did you turn? Wait, I thought this was me. No, it's not my name on it. What? No, that's management. All right, we'll let you do it. Okay, so how does CHS map to LBA? This is, and how do skip lists and remap lists work? That's what this is showing. Up above the darker blocks is cylinder head sector. Those are the physical blocks, geometric location. Keep in mind CHS is always oblivious to these defects. It has no idea that it, that they exist. That's where the translator will come in. The primary lists found at factories are P list, NRG list, maybe a few others. We're not entirely sure yet. Functions like this. So, zero, one, and two, those are good, those are good blocks, good sectors, and they map identical address wise to the CHS. Physical block three is bad, so we skip it and four becomes three, and that's how a skip list work. It's fast, it's significantly faster than remaps, which we will get to now. An important thing to note from his perspective is they have a limitation. They're static, so we tend to pair them with primary lists because they're expected to be static and unchanging. Physical, physiological via remapping, which is what the grown defects list are, so anything in G list is functioning like a remap. Again, zero and one physically, you know, having the similar, having the same address. We get to two and three, those are both bad. The heads will, you know, hit that and check the ECC, it doesn't match, it knows, translator tells it, okay go to the reserved area, which the heads have to fly to a totally different area. This can really, really start to increase the time that you have to spend, you know, waiting for your data to show up. So a lot of times when your drive's starting to slow down, it can often indicate via smart data kind of sort, I don't ever really rely on that, but it can indicate a grown defects list potentially. If you're in CS, this will cause it to resemble a linked list in a lot of ways. Yeah. And, yeah, actually that's all I got, but if you have more questions, you can talk to me about that later. I know it's kind of esoteric. Ah, science translator may be corrupted. Again, as I mentioned earlier, the drive will boot, it'll spin, or it'll do the partial boot, ultimately it's never recognized by the host. If you're able to get, you know, terminal debug mode, you will see these errors. Pretty much anything that begins with LED is indicative of a translator issue. Ah, that top one there ending in A051, it's just kind of like a classic stuck in busy. Ah, could, could be, you know, lists need to be merged or the G list needs to be purged. Ah, the second one there ending in A7E5, pretty much always means in a faro or moose drive that I'm gonna have to do the shorting of the read points in order to gain terminal access. Um, the SIM errors down here, any SIM error that you see that's like 1000 through 1008, probably corrupted defects list, don't really know which one, you'd have to research that or memorize it over time. SIM error 1009, pretty much always means corrupted G list, which is something that one of our tools can fix.  I came up with a plan worthy of a bond villain. It is fool proof, it cannot possibly fail. So we're going to corrupt a translator in a controlled manner because I like properly testing my software, run it through a manual fix, take some notes, take some numbers, model with a program, test it on some unsuspecting members of our hackerspace and profit. Who's written software before? Show of hands. Or played with proprietary hardware? Yeah. Yeah, you all know where this is going. Unsurprisingly, we ran into a few problems. So initially, we were expecting to find three lists, the P list, the energy list, which is that second form of P list and energy list, right? When we start splunking in the hardware, we find two out of three. So far, so good. We are on track. I'm feeling good about it. It's warm and fuzzy. Warm and fuzzy never lasts. I'm the reason I can't have nice names. So instead of finding one G list, we found three, which became our first problem. Which of these grown defects list are we going to try to overfill? And are they all really actually G lists? Because we just sort of found a bunch of lists and had to figure out what they were, what they did, and whether they were actually useful. So if you were confused by the remap, the physical to logical remap, ask me later what is copied and why that happens and how. All righty. Anyway. So we ended up trying to fill some of these lists and seeing what the drive did and seeing if we got the sort of signs of corruption we were expecting. We ultimately settled on the eponymous G list. So the first one at the top. And that's the one we used for the rest of our experiments and testing. So that's the list we settled on for future use. We also found these two. They're called slip lists. That means we found a total of seven lists here when we were expecting three. It's a little crowded in there. They aren't relevant to this talk and we're not entirely sure what they do, but we found them and we wanted to share that in case they're useful to other people. So we expected that we'd sort of fill the list by hand once it fall over at 2000 entries, we'd move on. That's not how that went. So initially, Mr. Dead here added 3000 some odd entries by hand. Not that she was counting. I added about 34 before wanting to throw out the wall and deciding pi serial was the lesser of two evils promptly scripted that. So even then I'm like, all right, we'll run it overnight, it'll be done. Well, we ended up north of 6000 entries and the translator wouldn't corrupt. And we're like, we don't think it has that much space. It shouldn't have that much space. We've run some numbers we think it should only have like two gigs. When you finally donned on me that the translator was actually a little bit smarter than I'd given it credit for and it was adding entries sequentially and just skipping the entire block. We're basically telling you this contiguous memory was bad and it was like, okay, contiguous memories bad, we'll just skip that. So I ended up having to add a randomizer to the script so it add random jumps so that it couldn't block it. Even then we found that the list had way more space. We were expecting it to fall over at 2000 entries and it usually takes between 4000 and 5000 for it to fall over. The drive didn't like being up for a lot of that. If you look at the far screen over there, you'll see the drive powering down in the middle of me doing things. So we found we also had to batch it and give the drive pauses between stuff. And you'll see on the left, that's the screen attached in diagnostic mode and it's got way fewer entries in its GLIS than I was expecting it to have, which was sort of my tip off that it was doing something smarter. I'm a little embarrassed to say it took me a while to see that. I just thought I was doing it wrong or wiring it back and it was a little awkward, which happened a lot. All right. So firmware is really weird. It was a third problem I personally ran into. What does level 6 E2 do? Any takers? This creates user batch file 0. How about level T, I4, parameters 1, you already know the answer. You can clean the erasers after class. This clears the GLIS. I have no idea how this corresponds to that. So if anyone ever says this to you? Every time you say code is self-documenting, God kills a kitten. Why are you ruining my punch line? I've been waiting 10 years to use this picture from my coworkers desk. Seagate firmware is extremely non-self-documenting and that can be very difficult to work with when you have to rely on partially transcribed Chinese manuals . I didn't think it needed a slide. We ran a difference between firmware, different behavior, different output. Some commands we expected output from didn't output in the Apple firmware version. It took us a lot of head banging to figure that out. When you are messing with this, you can't count an output to be consistent across commands. In some cases, the Apple version doesn't work. I'll try later and see if it actually did something. Again, that timing issue and why I wanted to automate this because it takes forever. This is just a little graph that Allison made. It's the results of several timing tests recorded during our attempts at the manual fix to short the points problem. This is from powering on the drive to powering on the drive. What's not shown is the timing between shorting the points and setting control Z. We didn't include that because we never actually got that far. As you can see, there's a huge variance. I think it was roughly between like 4.28 or something like that, up to like almost 6.4 or something. This is only from a single drive and we spent about 4.4 or something like that. We didn't include that because we didn't want to drive by overfilling the G list and here we are trying to figure out, how are we going to script this timing issue? We're still not there. We had some surprises. There were more lists than we anticipated. The translator was smarter than I expected by 4.8. I expected to be able to fill that sucker in 4 hours and it takes closer to 48. Data recovery technicians and professionals are patient when we're working and then in the rest of life we're kind of assholes, I think. Some other problems we ran into. We're only human. We're mere mortals. Software engineer can't do that. I was trying to use her script that she wrote to intentionally corrupt. I didn't read the documentation so I'm just like, go, go, go. Nothing's happening, Allison. I went back to manually inputting. That was pretty awful. I also poked through the PCB when trying to short the points. It's okay, I have so many of these to play with. The significant timing variants. The command output that we're waiting for in order to verify anything could take between one second. It could be 10 minutes even with really expensive data recovery tools like the PC 3000. Pie serial was less reliable. Don't use it on windows. I'm not the best Linux user at all but this forced me to learn a lot more and get away from windows. Don't judge me, please. Where we're at now, not that we've hammered at you for some background and our experimental progress and emotional journey, we can consistently reverse corruption if diagnostic mode is available. If you see some error 1100 9, we know we can always reverse that. That one works very well. We have good test environments. So if you want to play with reversing corruption, we can get you a drive that is in shape for that. The unfortunate thing is though is we haven't been able to reproduce the original target solution on our corrupted drives. Because of the variance in timing and the fact that I don't have a way to programmatically detect where it fits for the original problem. But that doesn't stop us from trying. So before we show you code, the development philosophy for this project is a little different. I generally believe that not all hackers are coders. It's a little bit of a controversial opinion these days in some circles but I do believe that and I have met some very technical people in the data recovery community who I would be so crass as to name names at a podium while we're being recorded. So this makes my hacker spirit sad and the whole point of this project is that the data privacy community has to there was not account of caffeine in that latte I'm saying. They have to operate with this giant veil of secrecy. Manufacturers are withholding information a lot of it's data. So our goal is to make this open and if you can't read the code then you don't really know what it's doing and you haven't learned. Code represents intelligence. So this code has been written with those people in mind. If you are a developer and you are a purest of your eyes because there are fragrant violations of dry it is over commented from developer perspective and it is very procedural and it will take a lot of time to do that because they are not my target audience I want people who do data recovery to be able to read it. It's in Python for two reasons one because it's the lingua franca of security and it's our poison of choice but also because in my experience teaching people it is the easiest type of code for non coders to understand. It is also GPL v3 because we are interested in making this free time adding to that so if you want to yell at me about the license choice feel free go for it. Please file your bugs with this philosophy in mind it is a little bit different. So to the meat of it what you are going to need to implement this at home with one of your ferro or moose sea gate drives side note I wasn't entirely comfortable with this and definitely very used to my open source so to do this entire thing if you already have the drive shouldn't cost you more than 15 bucks you just need a usb ttl adapter with the ftdi chip ft 232 access to the ground pin so you are going to have to shell your drive. If you have if you are not using one of the adapters that has shared power with the host you are going to need a power adapter you need the code which you can find on her github you will need to install screen to make sure you are connected to the hard drive and you can send these commands that you can cause corruption and reverse corruption. We also strongly encourage that you never use this on a hard drive with data that you give a dam about don't. You do have to match it in a very particular way. All hard drives are broken up into these bizarre families and lots of other things generally with the seagate moose and ferro you match the model number you can ID it if it is a good drive via the debug mode and I can document that somewhere there is a special command you can send for that if not you can kind of go off the model number ferro and moose drives almost always and in something something as with the model number. Match the firmware versions because again the exact specific type of corruption we were trying to fix with the shorting of the read points in order for the terminal access doesn't incur in a couple of like Apple it can occur there but it's difficult to fix because when it's seeking the service area and at what points it's reading the service area are different than in other firmware. The Dell specific firmware JC4A we were never able to get corruption I don't know why I don't know if it's somehow impervious to GList overfill I'd have to do a lot more research to figure that out. This is our setup that's my little pin read point shorting tool that I it's just tweezers and I tried to space them as perfectly apart to quickly hit the read points which are in the middle there and yeah it's just solder wrapped around the wrapped around there trying to bridge it it's a really shoddy setup and I'm embarrassed by it I guess and yeah those are the read points we're aiming for when we're right after power on listening for it to leave listening for the heads to leave and then once we actually did reach corruption because we have done that I just wanted to verify it in one of my fancy tools so it took up to the PC-3000 over there they have these fancy terminal connectors that are way better than way better than that much less likely to cause connectivity issues so yeah when you're doing this just make sure that you know you're crisscrossing your Rx and Tx from host to device it's like honestly kind of a kind of a trek to even figure out what pins are Tx Rx on the drives so Tx is always closest to the SATA then Rx right next to it and then ground is the very last so we didn't show the power connector it's there please be very careful there's a lot that you can do to destroy your drive and never get access to your data again so again if you're going to be doing this choose to start with a drive you don't care about at all and before you start very first thing back up the service area which yes you can do via plug mode commands you can save that way ask me about that later but definitely definitely definitely back up the service area alright so you've got your equipment you're ready to go please get your software installed in order remember to give it executable permissions hook up the connectors double check your connectors if you're in software have someone in hardware check your connectors determine what port your OS has been assigned read on the scripts read the I'd say read the documentation before you start but I'm a realist and know that nobody does that apparently so before you hit run please read the documentation because the order of the LBA addresses you enter matters so yeah running this once you get all your stuff and set up and are good to go is actually pretty quick however there is one gotcha I do want to mention so as we mentioned earlier especially when you get a full g-list your remap operations get really really slow and we did see some of the commands to add entries take north of ten minutes to fully execute so the script will time out you'll see received byte zero in the setup you can see on the left that we are actually getting the sectors reassigned I added to the g-list even though the script says we got nothing back this is because I went and made T and came back and there was the output so be aware that if you see receive zero you get an indication that the operation failed it just means it was taking longer than the script was willing to wait so don't freak if you see those alright so what's next this is the shameless plug for help we would like to try solving the original problem without shorting we'd like to write some code to configure the translator to rebuild based on the list you specify you may remember that we have seven of them to pick from I'm guessing you'll get some different behavior performance if you rebuild the translator with different lists we'd like to expand this to include other seagate families or even better other manufacturers could potentially do this with the ATEA commands because they are not specific to seagate so again long-term goal we would like an open source data recovery suite so that people don't have to spend an arm and a leg or hundreds to up to ten grand sometimes on data recovery feature requests again our current focus is on translators and defects list issues we do want to expand to include other solutions and of course other manufacturers code contributions just make sure you please keep Allison's philosophy in mind you know make this for people that don't know how to code we would love help testing that doesn't always include just writing scripts there's other things that I would definitely like help with and you can ask me about that later if you're interested open knowledge about the firmware all this again data recovery super super secretive the manufacturers are secretive other data recovery companies are very secretive you know it's hard there's a lot of digging there's some great forums out there HDD oracle is one of my favorite I highly recommend you visit that if you haven't so acknowledgements speaking of HDD oracle I really want to thanks build it I've never met him HDD oracle and he's just a fantastic researcher just wants to he loves what he does and he's not in it to make money and I think that's it and then I'd like to thank Leet Bunny and the two goons who sprinted over cross hotels to get us a projector at T minus five minutes because I don't think I could have done that yeah Leet Bunny is awesome he's always wearing a green hat pretty much definitely buy him food or booze or something and then bibliography you can find on my github that I never use or that little URL which is totally safe I promise and if something happens find Dean Pierce okay so we were supposed to take five minutes of questions but because we got such a late start we're going to take all of our questions over under the hardware hacking village sign to try to keep this room as on schedule as it is feasible to be at DEF CON so thank you all for coming thanks for forwarding the maze we look forward to seeing you over there to have a longer conversation