Hello, and welcome to today's Amazon Web Services tutorial. We will be covering a new concept in cloud computing, namely the serverless architecture. What does this mean? Well, essentially, serverless computing allows you to build and run applications and services without the need to manage the actual server. It also provides flexible scaling under heavy load and automated high availability, becoming a very versatile tool. The AWS serverless platform has many capabilities. As you can see, but for the purpose of this video, we will be showcasing the creation of a serverless web app from scratch. The structure of this project can be easily understood by following this diagram. All of the front-end code will reside in the S3 storage unit, which allows us to host the website publicly. Therefore, we start by creating an empty S3 bucket, like so. Give it a unique global name and click Create. And now to upload the static website's content into the bucket, I simply click Upload and drag and drop the build folder into here, like so. Note that I am using React as a framework, so it might be a bit different for you. The next step is making the bucket public, and for that we go on to Permissions and Bucket Policy and paste this bucket policy in order to allow public reads. Click Save and do not forget to change the bucket name. Now this bucket has gained public access. As far as configuration is concerned, the only thing remaining is enabling the website hosting in the S3, which will allow the objects to be available at the website endpoint of the bucket. Click Properties and select this card to do so. Check Use this bucket to host a website and input a root. For us it will be index.html. Furthermore, you can set up a custom domain for your S3 bucket by following the tutorial right here, which will be linked in the description. Anyhow, now you have the endpoint of the website. And if you click it, you will see that the static website has come online. Going back to the diagram, there are a couple of things left to do. We should set up an instance of the Amazon Cognito user pool, which enables us to register, login and authenticate securely. Then we will write the backbone of the serverless project that are the various AWS Lambda functions, which replace the server as they access and manipulate of Amazon DynamoDB, no SQL database, and respond to easy to use RESTful API calls. But first, the user pool. Head to Amazon Cognito and click Manage User Pools. Here you can create one or several user pools based on your application. For instance, one for your regular users and one for administrator. You give it a name and click Review Default Settings. Here there are various options, the password length, some type of restrictions for it. Authentication by email is a really important one, because you actually also gain a two-factor authentication while using the Amazon Cognito and finally create the user pool. You can find it under the user pools category and make sure you note down the pool ID that it has given itself, as well as the previous endpoint of the website. The final step of configuring the user pool is creating an app client for your project. I already have one in here, but you can always add multiple ones. Just give it a name and make sure you deselect the Generate Secret Client option, because that means that it doesn't support the JavaScript SDK. You can also manage and configure every single user in your user group by clicking here and for instance having the option to disable, delete users even, creating or importing new ones as well, and most projects use JavaScript these days. After that, leave the other options as default and click Create App Client. Make sure you save the App Client ID for future use. Once you have the two IDs, you can insert them into your application and benefit from the various Cognito APIs, like the login and register ones. Let us test this new functionality into the website. Now we can register a new user, for instance, with this email, a password that satisfies the constraints, and the verification code has been sent to the said email. It has sent this JSON using the user pool ID and client ID that we have just provided it with, and we have already appeared as an unconfirmed user in the user pool. After user verifies himself, he can also login by following the same procedure, and of course he would get a warning if his password is incorrect. Once Cognito is out of the way, all that remains is the serverless backend. AWS Lambda interacts with a database, so we must first create one. Head on to DynamoDB and click Create Table. Here give it a name and the primary key you can choose between various types, and then Create. For instance, here is my database. I have events with various categories. DynamoDB is a non-relational database, and you can edit the data inside at any time, as long as it is not the primary key of the item. For instance, here I have made a mistake, event capacity should be a number, and then click Save, it successfully changed it. Furthermore, you can customize it by adding new fields inside it. Let's say time of day, morning, and then it would add that category. Now that we have the database, we need to create a role, more specifically an identity and access management role for the future function, so that it has access to read and write to this database specifically. So let us do just that. Go on to the Overview tab and write down the Amazon resource name, so that we know which database we'll use. Then proceed to go to IAM and create a new role. Here choose Lambda, as that's the type of function that we'll use. In the Filter Policies search bar, begin typing Lambda, and choose AWS Lambda Basic Execution Role, so that it will have permissions to write to CloudWatch Logs, so that you can see whether you have errors or not. Then click Next, provide a name for it, and then you're ready to create the actual role. You can find it here, in my case. Now to attach the policy for our specific database, click here Add inline policy. For Choose a service, type DynamoDB, Actions, let's give it all the types of read, writes, and lists, and for Resources with specific option checked, go on to Table and click Add ARN. Here you can paste your own database ARN, and it will automatically fill in the fields for you. Click Add, and then you can review the policy, giving it a name as well. Once the role is created with both policies in place, we are now ready to create the actual Lambda function. This is easily done by going to AWS Lambda, and clicking Create Function. I like leaving author from scratch option enabled. For runtime, give it the programming language of your choice, a name, and a role. You can choose an existing role, and here you can see the one that we've just made, example Lambda role. Create Function, and the base one will be created for you. After we create the Function, you can see that it already has access to the CloudWatch logs, as well as the Amazon DynamoDB database that we've just created. The Function itself is a HelloWords tab, onto which you can build other functionalities. In order to test it, you click here, give it a name, and some parameters, if needed. In our case, it doesn't use them, so it doesn't really matter. Click Create, and again Test. As you can see, it says Hello from Lambda back. This is why we've added the CloudWatch logs policy. Now onto a more complex example. We go onto Functions and Get Events. This is a Lambda Function, which takes all of the events in the database we've previously looked at that have reached past a certain hard-coded date, due to this line here, using a DB Client Scan method. We click Test, and check out the results. Seeing that it does output all of the events. In the same way, you can make a post-event Function, which takes into account the parameters required to make a new entry in the database. Let's name it Test3, in this case, and click Test. The Function has added Test3. So let us see that. And here it is. Going back to the Function, a keen eye would have noticed that we have something called an API Gateway here as a trigger. What this does is it enables an URL, which we can use in our application, on which we call with a Get or a Post, for instance, and receive the result of the Function back in JSON format. This is essentially the final piece of the puzzle that connects all of the services together. In order to create one, remember, every Function has one API Gateway. We give it a name, a description if necessary, and an endpoint type. Let's choose Edge Optimized, in this case. Go to the API, go to Actions, create a method of the type that your Function is. Let's put it at any. Select Lambda Function, the region, and the Function that you've wanted the API for. Let's continue with the example Function, in our case. Use DefaultTimeout should be checked as well. You will land in this page, and the API Gateway is almost ready to be deployed. You will notice here in Method Request, Authentification is set to None. What does this mean? Well, if we change that, we can make it so that, for instance, the View Events, viewing all of the events in the database, is only allowed if you're already logged in. So after we do so, only then we'll be able to see the events. This is done by creating a new Authorizer. Click Cognito and give it a name. Let's call it User. Select the user pool that you've previously created, and the token source should be Authorization. Select and then back in Resources, click Any, Method Request, Authorization, Refresh, and you can see the newly created user. Now only being logged in into that specific Cognito user pool will allow you to access this specific Function. Those being said, we are now ready to deploy the Gateway API. You should also note this option here, Enable Cross-Origin Resource Sharing. I recommend you do enable it if you, like me, have opted to use Axios as a method of communication with the API, otherwise it wouldn't work. And finally, Deploy API. Select a new deployment stage with the name of your choice and clicking Deploy. This will yield evoke URL, which you will put in your application and onto which you will call with the get or post depending on the Function that you've wanted to call. The serverless web app functionality is now complete. You have the static website, sitting in the S3 storage unit with all the images CSS, HTML. The user pool uses Cognito to manage the login register and authentication, followed by the newly created database, which is populated and manipulated by the various Lambda Functions that you'll create, which are called from the project through the Gateway API. Thus, they all amounts to a single web application, which can be expanded upon easily. Last but not least, regarding price. Well, the cost of creating the serverless web app is null, as AWS accounts only for the time its service is actually run, which is mainly the amount of traffic and how many times the Lambda Functions compute. For me personally, it has been less than $0.1, and I have had the web app on for a period of about two weeks time, with extensive function tests. Ultimately, all of these resources are available in a GitHub project, and check the video description for more details. Thank you for watching.


Today I will be talking about unsupervised learning using dimensionality reduction and we're going to approach it from two different matrix decomposition techniques, namely principal component analysis and non-negative matrix factorization. And in the second part we're going to address it in the context of a chemistry example. So I'd like to start off by, for those of you who are veterans of this series, we're approaching this from machine learning by learning lots from data. As we've seen, we've been able to use cyber infrastructures, data repositories to develop predictive models and to conduct things like classification or various tasks. In this case we're going to address dimensionality reduction where we have an end dimension, our data lies in end dimensional space, maybe it's too difficult to be able to post-process or interpret, and we're going to try to reduce that down to a usable number of features that we can extract out. In this case we're going to do it in the context of reactive, decomposition of reactive materials. And again all these, all this is to help guide the design of certain experiments that you can do next. So as the outline we're going to go over dimensionality reduction looking at both principal component analysis and non-negative matrix factorization, and again address it in the context of a chemistry model. So I'd like to first start out by kind of dividing machine learning into two branches, namely the supervised and unsupervised branch. In supervised learning this is where we take inputs, pass it through a model, and our model will then help us predict outputs. If you've been a participant in our previous sessions you'll know that we've been able to develop linear regression models where we take in things like melting temperature to output Young's modulus or predict it. The second one, second part is the classification where we can do things like reading in different features of specific elements, say ionic conductivity or coefficient of thermal expansion, and then we can classify our elements into different crystal structures. In this session we'll be approaching, we'll be looking at unsupervised learning where we take in all our data that we've been able to capture, and we want to extract out certain features that stand out, and there are two sub-branches that lie under this category. The first one is clustering where you can see an example, this is an example of k-means clustering algorithm. They've been able to break up images of say the top row where we can extract out certain eyes or we can group things based on the hair or the hat, in the bottom row things like the road or the sky or the clouds. What we're going to focus on is dimensionality reduction, where we have a higher dimensional space, say this original image of the number 8 can be described by 784 components. We want to remove all the extraneous information, maybe some of the components don't really add anything, the ones we want to keep are say edges or vertices. And by reducing the number of components can we still retain the original image or explain most of what's happening in the original image by doing this dimensionality reduction, and this is very powerful as we can then post-process more data or do take this information even further. This is just to set up the motivation behind why we care to do dimensionality reduction. I'd like to start off with a very simple example of dimensionality reduction and kind of tie it into the algorithmic approach to principle code analysis. Some of you may be familiar with this, but this is a simple molecule defined by three atoms and it's just moving in time. So each of the atoms have their own motion, again three-dimensional so we have three n data that we have to take care of and it's evolving over time. We want to see if we can create a linear combination of unconcerned and uncorrelated motions from the time evolution of the left image. So what we know is we have a matrix of the positions, three n entries coming up with nine n being the number of atoms in our system and they evolve over time and this is how we can do a decomposition into again a linear combination of uncorrelated individual concerted motions. We can do so by defining or creating the covariance matrix. Essentially what the covariance matrix tells us is how one feature is correlated to another feature. What we want to do is actually decorrelate them. We want to have a linear combination of uncorrelated features. And so we do that by diagonalizing the said covariance matrix and by diagonalizing what we're trying to do is remove the off diagonal terms of our n by n covariance matrix. And this will help decorrelate the problem. And if you might have realized that this is simply a normal modes analysis, if you remember from chemistry or physics, and by diagonalizing said covariance matrix we can obtain two outputs, namely the eigenvectors and the eigenvalues. The eigenvectors are related to the normal modes. These are the concerted motions of the individual features and the eigenvalues relate to the frequencies. So it's just a simple example of how we can do dimension i reduction and this kind of follows the approach for principal point analysis, which I will put here. So we'll do PCA in terms of feature extraction for facial expressions. The algorithmic approach is shown here where we compute the mean of each feature, develop the covariance matrix using this equation, and then compute the eigenvectors in corresponding eigenvalues by diagonalizing said covariance matrix. In terms of the feature extraction, what you can see is that the eigenvectors actually give us useful information about what we've done. Remember what we want to do is define our overall data as a linear combination of uncorrelated eigenvectors. You can see that the first feature describes something to do with our face. It describes the most. We can see that it goes from dark to light in terms of shading. In this example here, this reference, and that in the second reference you can see that again we're able to extract out certain features. It puts together the eyes, nose, mouth, so on and so forth. So what I want to take away from this page is that eigenvectors encode features of our original data and that when you do a linear combination of them, you should recover your original data. So let's actually start going into the code. So I'd like you all to go to this link here, open up your browser, go to nanohub.org slash tools and there should be dimensionality reduction matrix composition. If you do so, you should be able to come to this screen here. I'll wait a couple seconds for everyone to catch up. So if you're able to follow along and put this in the web browser, you can click launch tools and it should take you to the landing page, which I have open right here. So what we're going to do first is address the example of principal code analysis. This will be our first tutorial. If you click on this, you should go and open up this notebook here. So we're going to explore principal component analysis by actually visualizing what it's doing. So if you follow along with me, full screen it. I want to start off by saying that these, if you have printed out the handout, these cells may look a little bit different than what you have there. And this is just for the purposes demonstrations of this demo, just to give more of a background and to provide more annotations to what we're actually doing with the code. The output should be the exact same, but for this demonstration, I just wanted to annotate more so you have a further understanding of what we're actually trying to do. So again, the PCA, what we're trying to do is emphasize variations or extract out correlations in our data set. In PCA, what I can do is actually help de-correlate our data. So in this toy example, we're going to have our two input variables. Of course, this algorithm can be used for more or higher dimensional and dimensions, but in this case, we're just going to approach it from two inputs. So in this example, we have a population where we know that there's some correlation between height and weights of individuals in our population. These are our two input variables. So what I want you to do is imagine sampling or even querying from 500 different people to obtain their information on heights and weights. And what we want to do is normalize this information from assuming a standard normal distribution. So these lines of codes are going to set up this normalize data already, but I just wanted to give you the framework for what we're actually trying to do. So we're going to eventually plot normalized heights versus normalized weights that has already been standardized. So first, we're going to import two packages. The first one is NumPy. It's a very popular computing package in Python. And the second one is Matplotlib, which is, again, standard visualization. So in this first line of code, maybe some of you are not familiar with it, but what we're doing is we're calling the NumPy module. We're going to load the random class and we're going to define a random state. You can read about what it does here. And if you're interested, you can go to the documentation to read up more about it. But all we are trying to do is develop random number generator so that we can use this random number generator we defined to draw from a variety of different probability distributions. And that'll be shown below. We're just defining a random state. From here, we're going to find two different arrays. Our first array is of shape two by two, two rows, two columns. And from here, we're going to again call our random class and using said random state we defined as one. We're going to pull from, we're going to randomly sample from a uniform distribution over the range, zero inclusive, one exclusive. That's our first two by two array. The second array we're going to define, which we're going to push to our variable B, is we're going to have a two by 500 array now. And instead of pulling from a uniform distribution, we're going to now pull from a standard normal distribution that's defined by the N. That's what it says here. Then what we're going to do with these two arrays is we're going to dot product. Namely, if you remember your matrix algebra, when we do dot product of A comma B, we're going to eliminate these variables and we're going to end up with a two by 500 array. And what we're going to do is do the transpose of that. That's the capital T. And at the end of the day, we're going to be left with 500 one by two arrays. And so if we run the code cell, again to run this, you can do shift, hold shift and hit enter, or you can hit the run button. If we run this code cell, you should see that I've shown the first 10 entries of our data, which is 10 one by two arrays. So this is our normalized heights and weights going from say a random sample that we've defined our 500 population, normalized heights, normalized weights. And so what we're going to do is we're going to then call matplotlib and we're going to produce a scatter plot of our normalized heights column and normalized weights column. And we're going to set up some color scheme for the grid, make sure the axes are normal just for visualization aspects. It's all status. And so what you can see here is that in this problem, what we can see is that our normalized heights and normalized weights has some sort of linear correlation. You could say there is some relationship between the normalized heights and normalized weights of our two inputs. Now, this problem was set up in such a way such that we don't see any points in the top left or bottom right corner. That's because PCA pulls from linear algebra. And so it's really not able to handle non linearly distributed data. And it's also not good at non multi Gaussian data distribution. So as we frame this question to succeed, we want to be able to show is that if our PCA algorithm can learn this linear relationship between normalized heights and normalized weights. So let's move on to the second cell. We're going to be able to try to understand what the PCA algorithm is doing by actually interpreting what it's outputting. So first, we're going to call the PCA class. And this is from the scikit-learn.decomposition module. If you would like, you can look up more on it. It has a lot more information than what we'll go over, which is PCA and NMF. But we're going to use this module for the purposes of this workshop. And so we define our PCA object and it has a variety of arguments. What we're going to define is that we're going to say, let's have let's explain our data using only two components. Very easy. It's two-dimensional. So we're going to have two components and we're going to pass it to a variable. And what we're going to do is we're going to fit our object to the data. Fit the PCA model to the asset X as defined above with set argument and components being two. And we're going to print out what the object is actually doing. So if you run that, you can see that what we have as an output is we printed out the PCA object, a bunch of arguments that we haven't touched. The only one that we're interested in is we modified this argument to be two components. Okay, great. What are we actually doing? What is the fit actually doing? And so what's interesting is that the fit can learn some quantities from the data. These are going to be defined below as components and explain variants. And I will go over it real quickly, but I think it'll be easier to visualize. And so as I stated, we're fitting our data to two components. And so each component is defined by a one by two array. So this will be the one by two array for component one, one by two array for component two. I'll go into actually what this value, what these actually mean in the next cell. But I like to move on to something that's more interpretable, which is the explain variants. Again, this is the explain variants for component one, and this is the explain variants for component two. What this is saying is that if we do a two component fit, component number one, however the algorithm imputes that, is able to explain a little bit over 75% of all the variants in our data. And component number two can explain just shy of 2% of the variants in our data. And just for completeness, I've done a simple sum of them. And what we're saying is that if we do a two component fit to our data, the two components in total can explain 77% of all the variants as defined in our data. And so what I want to, we'll actually understand and visualize what this is doing. Hopefully you'll get a better understanding of the components explain variants in a more visual approach. What we're going to do, take it from me here, is that we're going to use the components to define the direction of a vector. This is a vector of two dimensions. And we're going to use the explain variants to somehow correlate that to the magnitude of the vector. So the components are going to find our vector in terms of the direction. And the component variants is going to explain the magnitude or the length of the vector. So that's what this line of code, this code cell is going to be doing. We're going to go through the code and we're going to create an arrow based on what we read in. And so this defines a definition to draw the vector. All we're doing here is defining dictionary. This is just the style of the area that we want to visualize. This is our arrow style, the width of the lines, the color of the lines. And then we're going to use this command to just put the arrow in the figure with the defined style. Arrow props is the arrow prop here as defined. Again, we're going to plot our scatter of our original data, normalize heights, normalize weights. And this just sets the transparency to 80%. So it'll be very light color. And then we're going to loop over our explain variants array and our components array. And we're going to define a vector v and just draw the vector v on top of it. So again, we're going to use the components and explain variances of our fit defined above. And we're going to overlay that on our data to actually see what it's saying. So if you run the cell again, hit run here or shift and enter. You'll see that we're actually able to visualize our data. Again, light blue circles shaded, that's our original data. But then we've projected, we've been able to develop our fit for the PCA and project it onto our original data space of normalized heights and normalized weights. What you'll see are two vectors, this being component one and component two. How is that? Well, because I remember component one explained the most variance, over 75% of our data, therefore it should have the longest arm. And if you look at the actual entries to the matrix of our component one, you'll see that it kind of has the value related to the slope. So both of them are positive, positive. And in the second component, explain variance is a lot smaller and this has a negative positive value in the explain the components. And so what you can see is that when we do principle cone analysis, it creates the principle axis along which we're projecting our vector on. And it ranks it such that the most important, all the variance that is explained by the most important component comes first. That's principle component one and this is principle component two. And it shows how important we can project our data in terms of the variance onto these principle axes, one and two. Okay, so now that you understand or get a feel for how the PCA algorithm works, again it explains the variance, the overall variance of our data that's most explained by two components because we did a two component fit. Let's go down to actually reducing dimensions. Again, we can only really go down from two dimensions to one dimension, but I think it'll give you an idea of what's actually happening when we do a dimensionality reduction using PCA. So again, we're going to develop the model. In this case, we're actually going to now define our object with only one component, right? Going down from two to one is all we can do. And we're going to call our object that we defined as this variable. We're going to call the object and we're going to do a fit underscore transform. Now this might seem a little bit different from what we did previously. Remember previously we did something that's a Kim to step one. We've taken our object and we just apply to fit. So if I run this code cell, remember shift enter, we just define the object, which is doing, again, in this case, we have our argument as one component. So that's what we define our new object as one component. That's all we've done in our fit. What we want to do is we actually want to manipulate our data. We want to actually show the output of our data after we've done the reduction from two dimensions to one dimensions. So we have to do that by using calling the transform option in this object. And if we call this transform function, you can see that we've now gone from a 500, one by two arrays to 500, one by one arrays, right? There's only one value that's obtained because that is the single component we defined. All this is doing is we're doing fit and transform in one step. That's what fit underscore transform does. It's doing one step. So now we've actually reduced three dimensions. Let's actually see what it's done in terms of transferring data. You can see that we started out originally, remember, 500 by two data, normalize sites, normalize weights. Now we've done a transformation to drop it down to one component, 500 by one, whatever this one component is defined by the PC algorithm. You can see that this component is defined by only a single one by two array. This is the component for the principal component that explains the most variance in our data. And again, this has an explained variance of 75%. So while dropping from two dimensions to one dimensions, we can still describe over 75% of all the variance in our data. Pretty impressive. Okay, so now that we have our transform data, this is again in principal component space. It doesn't really mean anything. We've just projected our data to principal component space. So how are we actually going to be able to visualize it and compare it against our original data? Well, this requires us to transform it back into the original space. This requires us to call the specific function inverse underscore transform. Remember, XPCA, which is our variable, is just what we've done to do a fit in transformer data into principal component space. What we need to do now is do the inverse transform and put it back into our original space. And that's all we're doing. We're going to put our fit back into the lower dimensional space. That's 1D. We need to map it back into our R2 space. Again, normalized heights, normalized weights. So we can plot it against original data. You can uncomment this print statement if you would like to see how that actually what the outputs will be. But we're just going to plot and plot the scatter points of normalized heights, normalized weights. Again, setting the transparency to 80%. And then we're going to plot our new data again because we've mapped it back to our original space. We have two dimensions. So if you run this cell, what you'll see is that our original data again is in lightly circled blue. And that we now have our new transform data that we've reduced from two dimensions to one dimensions. And it's all projected onto this single principal axis. The one principal component that if you remember, the vector looked like it had its initial point here and it was pointing this direction. This is the original vector projection. So what we've done is we've just mapped all the variants of our second component. We've gotten rid of it and we've projected it onto principal component that explains the most variants in this case because we only have one component. This is what going from two dimensions to one dimension actually does in PCA. So hopefully it gives you an understanding of how we can actually, how the PCA algorithm works. And we'll actually see it in the second example, which is our chemistry model. So with that, I will jump back to the PowerPoint presentation. So as I stated, we've done PCA dimension reduction where we took our two-dimensional data, which was in normalized heights, normalized weights. We applied PCA and we projected onto one dimensions to put it back into our original space. And so as you can see, we're actually able to successfully learn linear correlation. Of course, this is all set up so the example would succeed. So let's actually put it in a real example, which is our chemistry model. Before that, I'd like to go over, of course, what NMF does because we're going to call our second method. Now we're comparing PCA and non-negative matrix vectorization. In the interest of time, if you'd like to go to this paper here, it defines both the methods, but we're going to kind of provide a review of what the paper actually does. So in this paper, the authors have an original image of a face. They wanted to define it. They want to see how PCA and NMF are able to recover or what it actually does when it does the combination to create the face based on the dimensionality reduction algorithm. So we'll first start with PCA. What you can see is that PCA, which is the product of two matrices, can recover the original image. Namely, the individual matrices are the principal components or the bases and the weights matrix. And you can see that they took the principal component of this object here. When you do the dot product of the weights, you can recover the original image. What you'll notice is that their weights matrix, remember this is a linear combination and all they're doing is diagonalizing the covariance matrix, will allow for positive and negative values in their weights, red being negative. So if you think about it for a minute, that doesn't really lend itself to interpretability. What does a negative feature actually mean? Some of these you can see have red eyes for these faces or red cheeks. That doesn't really mean anything physically for this eigenvector. So that's not very interpretable. So what they then did was move on to non-negative matrix factorization as an alternative method to do this decomposition process. And what you see is something very interesting. Again, we have our components or our bases multiplied dot product with our weights matrix to recover something that's very similar or approximate to the original face image. What you'll see is that we don't have any negatives, right? All the values in our components and weights are white or black, namely zero or greater than zero. And now we're actually even able for this eigenvector to actually extract out a specific feature, which is eyebrows. And this is very interpretable. So this lends itself to why NMF can be more interpretable than PCA. Let's move on to our chemistry example. So in the chemistry example, what we have here is complex chemistry occurring in our system where there's interactions with atoms in its nearest neighbors, things are bonding, things are breaking, chemistry is happening, our system is decomposing, it's a reactive material. What we want to be able to do is to deduce a reduced-order kinetics model from set simulation or data that you captured. In this case, if you go back to basic chemistry, we want to just simple curves that explains the decomposition process of our complex model. In basic chemistry, we can say is that we have reactants as our first curve, evolving into intermediates, and finally, products. Three simple curves that explains the overall time-dependent behavior of the chemistry happening in our system over time. And with this information, we can then do fits to develop rate equations. And that'll be important because then we can pass this information into say mesoscale or continuum-level models for them to take it a step further. So you can see the importance of dimensionality reduction in this context of the chemistry example. So how do we do so? Well, a naive approach would be to look at all the atoms. We have their information about positions, we have information about their nearest neighbors, what they're bonded to, so on and so forth. We can track all that information evolving over time. You can see that that will vastly blow up as our system grows, number of atoms grows, or the number of time frames we have to track grows. This is a higher dimensional space which we're not going to easily be able to track. So how can we use dimensionality to our advantage? Well, we notice that we can decompose our overall chemistry into a finite number, 280, different bonding environments that explains the overall decomposition process of our system. With this finite number, this gives us a cap on the number of features we have to track over time, 280. And what you see very interesting is that there are certain similar shaped curves to what we would like to have as an output, curves that go down, namely similar to v1, curves that come up and go down, which look like v2, and curves that come up and stay, which look like v3, our products. So this is what we're going to do with our example, using intuition about chemical decomposition products from very simple chemistry. We can map or project our overall complex chemistry into a usable format for which we can then apply the PCA or NNMF algorithm to. So let's get started. So I would like all of you to go back to the landing page. I hope you didn't close it. But if you did, go back to the link and open up the second notebook, PCA and NMF. I'll use it here. So back to the landing page, go to the second notebook, PCA and NMF. And if you click on it, it should open up this notebook. So again, the overview just reviews what I stated in the previous slide about the overall chemistry. PCA and NMF, I'll address this later on, but you're more than free to read it on your own time. Let's get started with the first code cell. What I want to show you here is how complex our data is. So if you run the cell, remember again, shift, hold shift, hit enter, or click the run button at the top. It should print out kind of the format of our data table. Part of the codes, we're going to use the pandas library. And we're going to do read CSV as our function, reading in the data file that I defined. And we're going to skip some rows as the header, and then we're going to show five rows. So this kind of shows you the format of how our data is defined. And this shows you the first rows in information. Again, all this is a show that's very complex and you're more than welcome to look at it on your own time. What we're going to want to do is this code is going to take about 40 seconds to run. Let's just run it, and then I'll go over kind of the breakdown of code. So shift enter, should run, go to the very top. And so what we're doing is we're essentially trying to manipulate the information I just showed previously in a more readable format, where we can actually understand and capture the number of the bonding environments. Remember, I stated there were 280 for each of the atoms that it experiences over time. So you might ask yourself, how did you come up with the number 280? What does that actually mean? Well, we know that for CHNO atom types, this is organic reactive material. There is only a finite number of combinations. And we're going to go back to basic chemistry and assume that we have a maximum of four bonds. Shown here are various, using intuition, various molecules that you can look at that we know that exist in our decomposition process, just shown as an example of the different bonding environments we can have. So what we're going to do is import numpy. We're going to call all the functions in norpy. We're going to get sidepy, mat, sys, and read. This is just for a string comparison. Now we're going to define our arrays. Just going over basic code for those who are not familiar with Python. What we're going to do is first define our function here, where we're going to call a variable ifile, and what we're doing is we're just opening the input file I defined above. And we're going to read what's in the file. Now we're going to loop over all the lines in our file. Now we're going to start splitting or delimiting on white space. This is the white space character. And if the second index is equivalent, exactly equivalent to the string time step, then we're going to pass the third index to this variable called tstart, and we're going to make an integer. So you might ask, why is this the second index? Why is this the third index? In Python, we start counting at zero. Simple as that. So that applies for loops, applies for conditionals, applies for arrays, applies for strings. So after we reach the point of looping through, also our second conditional will say, hey, look for a number. How many atoms do we have in our system? And we're just going to exit out of the loop. And then we're going to close our file. So doing so, we're going to move on to this code where we're just essentially going to, this is a very single line way of counting the number of lines in our file. It's not cheap by any means, but you can compress it into single line. We're going to open our file, start counting up the number of lines by summing, and this will pass it here in the lines. So if we go down at the very bottom, you'll see that we've been able to complete one of the bond table. One thing I'd like to point out is that if you're doing what we're going to do in this notebook, which is writing to a file and then reading it immediately, sometimes the script can't catch up to what you've printed if you jumped immediately to the next code cell. So what we want to do is make sure you flush the data from the buffer. That's all this command is doing. It's very useful. So we'll move on to the second code cell, which is after we've done our post-processing in a meaningful way to be able to do some visualization, let's actually look at how we've done the post-processing. And I've shown you this image before in the slides. But again, we're looking at the 280 different bonding environments, evolving over time. We can see that there are three main features that we can extract out. So that gives an indication to say, hey, let's use end components being equal to three as our dimension-high-reduction number of components. So we'll do so. First, we'll run PCA as we've seen that previously. Again, remember we have to develop and create our model. So we'll import the PCA object from the SK Learned Decomposition Library. We're setting number of components to be equal to three. And then if you remember previously, we've done a lot. We only defined the number of components as three. In this example, I'm just defining a lot more other arguments for an object just so you can actually reproduce it, right? One of them is a random state. If you do a different random state, you're going to have probably a different solution you run every time. So we're defining our model, and then we're going to fit our model to the data. Remember, this is our 280 different bonding environments evolving over time. Those are the two dimensions over our array.  What we're showing here is what we're going to do is we're going to create our model, and we're going to do that fit, underscore, and transform. Remember that two-in-one process. And then we're also going to show our components. So shift, enter. What we're going to be able to see is that, again, three-component fit. Using PCA, we can explain 93.3% of all the variance of our 280 features evolving over time by just the first component itself. The second component gives about a little bit over 5%, and the third component can only give 1% worth of explaining the variance of our overall data. And this provides us with a cumulative variance of 99.6%. That's very impressive. We're able to retain most of the variance explained by our data with only a three-component fit. This is great. So now let's actually visualize what the components themselves are doing. So if you run this code cell, you'll see that we have our three components. Component 1 explaining the most variance, Component 2 explaining the second most variance, and Component 3 explaining the least variance of our three components. Now, you might think to yourself, hmm, this doesn't really look like the concentration profiles I'm used to understanding. Concentration is probably, it's always zero or positive. There's no way you can have a negative concentration explaining our data. So you might say to yourself, since concentration can't be negative, these immediate results are probably not interpretable. You might come up with a fancy way of manipulating the data, but for all intents and purposes in this demonstration, we're going to say that it's probably not interpretable. These outputs are not interpretable enough for what we want, which is concentration profiles to explain the basic chemistry decomposition process. Now we're going to move on to non-negative matrix factorization and showcase the power of what it can do. So, we're going to, again, start off with NMF and develop the model, a very similar PCA. Now we're just importing the NMF object from sklearn.decomposition class. We're going to, again, define our number of components to be three. We're going to set up our model such that we have, again, three components using the NMF object. Now we're going to initialize exactly the same as PCA so that this is a one-to-one comparison of what we've done previously. Again, we'll do fit underscore transform for our model and we'll define the number of components. If you do ship enter, you should see that we've completed doing our NMF fit. Let's actually visualize what the fit is doing. You think we'll get something similar PCA. So now we've done the fit. You can see that now it seems to be a bit more interpretable. Now, our constraint NMF is that our values in our array that we decompose have to be greater than or equal to zero. This is a constraint that you see here and it looks more interpretable. You can see that this looks very similar to our reactance curves going down. This looks very similar to our intermediates curve coming up and going down. And finally, we have our products curve coming up and staying up. However, you notice that unlike concentration, this isn't set in the range of zero to one or zero to 100%. We're going to have to do some normalization on this data. So this code cell, all it does is it does the normalization aspect and we'll plot it. And now you see we have something that's a lot more interpretable in terms of our final goal. Remember, our final goal is to get three curves that essentially decomposes our data down, three simple curves that can explain the overall decomposition process of our system, namely the red component one, equal to reactance, blue curve, component number two, equal to intermediates, and green curve, component number three, meaning that as products. And so we've actually been able to visualize this and we see that it follows a very similar scheme on here. So with that, I'm going to jump back over to the slides. Now we've again applied dimension reduction using NMF or NNMF, non-negative matrix vectorization. And what we can then do is now that we have these concentration profiles, we can go back to our chemical kinetics from chemistry and fit them. Next equations. And what you'll see is that we've done that for ensemble temperatures, different initial temperatures, and we can get overall kinetics parameters. And then again our end goal is to do this dimension reduction to get kinetics parameters that are passed into continuum or mesoscale models and for their benefit. So we've been able to reduce our end-dimensional data into three easy curves to get values to pass further along. So I'd like to go into the non-negative matrix factorization algorithm a little bit to give you a flavor of what's happening. So when we do our factorization, what we have is our matrix V. This is our encoded matrix. And we want to factorize it into a dot product or approximately equal to, as best we can, the dot product to some matrices, W, our features matrix, and H, our coefficients or weights matrix. What you'll usually notice is that in our decomposition process, we're going to have to define a new variable P, say our number of components as we've done, two or three in the previous examples. And this will usually be a smaller dimension than our M by N encoded matrix. This allows us to be able to reduce the number of features and only look at the most important ones. So what makes it non-negative? Well, the non-negative aspect of this matrix factorization technique comes from the fact that we have to constrain both our features matrix and coefficients matrix being non-negative as in the entries or the values inside our W and H matrix all have to be greater than or equal to zero. And of course that means that our encoded matrix itself has to be non-negative. So a common approach to actually doing and applying this constraint is to minimize this error function, which is the difference between our encoded matrix and what we would like our features matrix times coefficient matrix to be. Doing this difference, minimizing the Frobenius norm. If you're interested in pseudo-code and how we're actually able to update W and H over the number of iterations in this minimization scheme, I would urge you to please look at this paper here. They go into very great detail about the pseudo-code and actually obtain W and H. So I'd like to summarize PCA and MF overall. So with principle component analysis, we saw that in the example of the facial features we're able to get both positive and negative weights associated with the face. And the constraint on this is that we ensure the eigenvectors are orthogonal one another. Remember, it's a linear combination of uncorrelated features. What this does is though is that with this constraint it does a global transformation, right? We had some of the features that had both noses and eyes and mouth. So it's a more global transformation. And each of the components are defined by the largest variance, or largest explained variance that that component can with respect to the original data space. So each component explains a certain amount of variance of the overall data and it's, they're ranked by component number one explaining the largest variance. When we go to non-negative matrix factorization it's more specialized because what we're allowing is only added to combinations. Remember, the coefficients, both the W matrix and the H matrix has to be, has to contain non-negative values. So this is a hard constraint that makes it non-negative. And because of this constraint we now have weighted parts face features. Remember again back to the faces that the matrix for the coefficients or the weights was black or white. That means the values are either zero and this means that when you do, when you sum them back up again to recreate or recover the original image it was very part space. We were actually able to extract out specifically the eyebrows or the nose unlike PCA which kind of mixes the features together to do the linear combination. And so in this case, NMF actually provides more interpolate outputs in the examples that we've covered here. So with that, I'd like to thank you for sticking around for the tutorial going through Dimension I reduction and going through principal phone analysis and NMF. And I'd like to bring you back to kind of the whole grand scheme of what we're trying to do. Remember, we're learning from data. We have cyber infrastructure, repositories and cloud computing to help us assist in this. We've done, we've created predictive models, done classification in this specific session. We've looked in great detail at Dimension I reduction both PCA and NMF and all this has helped guide in the design of experiments. So with that, thank you for your time. And at this moment, if you have any questions you may unmute yourselves. Feel free to ask with remaining time left in the session. So thank you. Thanks, Michael. So I'd like to ask everyone if you could unmute your mics and give some appreciation to our speaker. And we can start with the Q&A. Okay, so I'll start with some of the questions from the chat. So one of the questions that we saved is how does one interpret the reduced features obtained? Well, is that how does one interpret the... Reduced features? Like the components? Are they asking in the context of PCA or NMF? I believe that would be PCA. So with the reduced features, that would be the eigenvector matrix, right? So this is the features matrix, not the coefficients or weights matrix, just to clarify. So for the features matrix, what this is essentially saying is this is the eigenvectors in our decomposition. In terms of the faces, this is what we're able to, the algorithm breaks down to predict the most variance that explains kind of the eyes, ears, nose, whatever in our overall image. I'll use that as an example. That's probably easiest to understand. Let me go back to this example. What you can see is that our original image is a face. The eigenvectors define kind of the amount of variance or how much information is breaking down pixel by pixel can be explained in this decomposition process. So you can see that the latest eigenvectors really don't explain much. In this case, it's just explaining dark to light shading and doesn't add anything, whereas the highest eigenvector explains the variance. So in my interpretation of this, it's getting encoded information for the features by essentially doing a linear combination by breaking down to linear combinations of uncorrelated information. Maybe I'm just regurgitating what I said along the slides, but this essentially just provides information about certain features altogether. All it's trying to do is solve the eigenvalue, eigenvector problem by doing this decomposition process, diagonalize the covariance matrix. So it wants to de-correlate each of the individual features or, say, certain colored pixels in this case, pixels for the eyes and pixels for the nose. You can see there's slight differences in colors. I'm trying to break that down as best as I can by the algorithm, and it just says, this is what we can get for the feature number one, something to do like that. So maybe that didn't really answer your question, but it essentially tries to extract out certain features but ensure that we're applying this constraint to our system, which is each feature itself needs to be eigenorthogonal to one another. Maybe someone else in the chat who's done PCA can also provide some input if they have... Do you have any comments on that? Okay, we'll see if there's any follow-up if you need clarification for your explanation, but let's continue. So for PCA, do we assume that the inputs would follow a Gaussian or normal distribution? Yes, I believe so. As I stated previously, we have... I forget where I put it, but in the slides, it has to be... This method works best assuming a linear combination. So it's pulling from linear algebra with this matrix decomposition technique. So it's best if the data itself is linear correlated. In terms of, I think you said Gaussian, that would also help. There are other techniques that I haven't covered here, which rely on PCA method, but it can be used for data that may not be linear correlated, is the best example I can have. These are things called kernel PCA. If you can go back to the scikit-learn decomposition website, you can look up things like kernel PCA. This is very useful for data if it's in, say, a ring, or there's a large spread, almost looks like a target. It can apply the PCA algorithm to this multivariate data space that may not have linear correlation. So that's maybe addressing the question that's another constraint of PCA that I found online. And of course, there are a lot of resources that explain the downfalls to PCA. But I thought I'd frame it in examples that people will use in this, which is facial recognition. PCA is very prevalent there, and you can do a quick Google search for different examples. Okay, so another question. In the scheme of machine learning, going from data to predictions, where does the dimension reduction fall? So dimensionality reduction will help if you have so much data that you can't even make a prediction, right? So say, for example, you have data in, like, a 10th dimensional space. Maybe that's interpretable for you. So you don't have to do dimensionality reduction. Let's say you have data that's in, like, the 1,000th or even a millionth dimension, things that you don't even know that there's information there, right? This is feature extraction or abstraction. We're trying to just pull out comments or abstractions. We're trying to just pull out commonalities in our data. We'll pull out things where we're fogging with another. If you have too much information or you want to try to find patterns in your data, it's very useful to do this to pull out those patterns that may be then something that you can say, oh, hey, look, there's actually a relationship between Fed feature one and feature two. Let's put this in a model. So I would say that it's very useful for your data if it's in a hierarchical space that's too complex to even post-model. I would also add that you can use this data at the back end as well. You don't have to do it at the very front, but you can do it afterwards. If you run it on a model and you obtain some outputs, maybe then it's also still not interpretable. So you can then also apply dimensionality reduction to make it into a meaningful interpretable results. So it can be used essentially anywhere, I think. Okay. Another question. How can you find redundancy in your remaining components? How can you find redundancy? That's a really good question because it's kind of the information that you throw out. That's a really good question, I think, though. That's some more feedback, but I would say that I haven't really addressed redundancy, I think. In this case, again, we're trying to break down our data into so many eigenvectors, in this case, for PSDA, just to recover the original image or the information that got discarded we're not really concerned about. You can see that even in this example here, we've gone to the 500th eigenvector, and it's not really giving any more information. Outline of the face doesn't really do much. Maybe you can stop at, like, say, the 10th feature here or the 50th feature. In terms of redundant information, maybe that's useful information that I haven't thought about, and it's redundant. Maybe you can decompose that even more, but it's not something that I have looked into in further detail. Thank you for the question. It's very thought-provoking. I think this is the last one from the chat. How many components or how do you choose how many components should be used in both PCAN and non-negative matrix? With any reduction algorithm I would say that there's really no... It depends on your interpretation of what you want to do. In our case with MMF, putting in context, again, the chemistry model, it's very easy because we have intuition. We're pulling from chemistry and we know that this is what we have to do. That allowed us to say, hey, let's go with three components and that's what I'm going to fit with, by all means, we're able to obtain three component fit and it looks very close to chemistry. We could easily go into four components and that's currently in the works. We can develop models for that. I would say that with PCAN and MMF you would probably want to look at the cumulative explain variance. There's no real good metric for defying that. That's still a research question of interest, but I would say if you can explain say 99% of all the variance in PCA or you can project your data in MMF on to say five components and still explain each of those features overall, I would say you've retained most of the information, most of the original data that you've done by projecting down to this lower space. Again, that question about what is the correct number is still to be determined and is still a metric that people are looking into. When is enough enough? Sorry, I couldn't answer your question, but if you come up with the answer please do publish it off the community. Okay, another question. For numerical inputs and outputs can I identify significant input parameters from this decomposition? Input parameters. So what you're saying is like if I I wouldn't know if you can, from these examples it's very difficult if you can extract out input parameters I would say it's more of a method that I've shown is how to find commonalities for correlations and break apart these correlations in analysis but if I remember back to the PCA example, we went from normalize heights and weights instead of solid reduction, but it's not like we could say that normalize heights was what we should focus on those normalize weights or what we should focus on those we're able to project a dynamic to do a simpler dimension of 1.2d and 1.2d I think it can give you an idea of maybe what, how I should answer this is if you do this dimension reduction going from a higher order dimension to a lower dimensional space, it might be able to break, it might be able to dissociate itself from certain numbers of those dimensions, which are attributed to certain number of inputs as you were suggesting so maybe there's a way that it's able to know to collapse say going from 10 features to 3 features it could collapse maybe 4 of those dimensions and it projects it only to the remaining 3 dimensions as a linear combination of those 3 and you can eliminate the inputs essentially that explain the least amount of variance so in a way I think it's possible but I haven't shown it here so I don't want to claim anything inclusive Okay, can I identify local variances using PCA and MF rather than global variances? So I know PCA is a little technique I haven't looked further beyond than just the vanilla PCA I'm sure that there are other techniques, I think there's one called independent ID or something like independent decomposition that can probably do local comparisons. I know that MF is more local than PCA but again the constraint is that the data you read in has to be negative as you pause it I would urge you to look beyond PCA and MF this was just to give you an understanding of decomposition matrix decomposition I didn't really answer the question but I'm pretty sure there should be methods out there or any development or individual or independent decomposition Okay, I think you can see the last two questions in the chat I saw about tensor Yeah, what's the difference between non-negative and tensor factorization? So to my knowledge there is a paper that came out by a collaborator that did non-negative tensor factorization so I'll explain in that context what they were able to do is instead of non-negative factorization which you saw was a 280 by time matrix they broke that down into different they broke it down into different combinations I would say I have to look back carefully of the 280 over time they essentially applied it as a tensor instead of a matrix so they added a third dimension instead of a two dimensional object they had it as a three dimensions and they put it in tensor and they were able to essentially define different iterations of their data by applying this NMF I think they evolved over different seeds and they used this tensor factorization technique to to define a metric that best defined your chemistry scheme say for three components remember we did our random c to zero but you obviously get a different answer if you did random c to one or random c to the thousand so they're able to use tensor factorization to have a third dimension which is a different number of seeds and in doing so we're able to do decomposition to extract out the best of their approach I think this is one way of using tensor factorization over matrix factorization maybe your data is sensitive to the seed for some reason but to my knowledge that's how tensor factorization works and I know that there's some other algorithms that do tensor factorization again you can look up on scikit learn and I believe MATLAB has its own modules to import for matrix decomposition but to my knowledge that's how those collaborators use tensor factorization here's a quick question under what circumstances is PCA preferred over non-negative? I mean it's almost and that's all we're all aiming for I think for most data that has positive negatives if you can have a negative value in terms of a feature if you just want to have extract due to decomposition such that your outputs are orthogonal or linearly independent from one another then you should do so it's a pretty strong technique and MF just requires that the data has to be non-negative and it does this decomposition I believe that there is a technique out there or there's further approaches that actually combine both PCA and NMF so NMF in itself is not completely orthogonal there is some correlation to the vectors you can't say that component one is completely independent from code 2 doing NMF there are people who have combined both PCA and NMF to do both I think they do NMF first and then they apply the constraints of PCA to the orthogonality on top of your NMF data so there are things out there that can do both of them just haven't been implemented here but I would say the strength of PCA is that it ensures that your components are orthogonal to one another so there's no reason for correlation between the two okay I think that was it so again thanks everyone for coming if you can let's thank Michael again for this lecture and thanks for joining us for this the hands-on tutorials for machine learning it's been a pleasure thank you


Your coca-cola bottler presents Claudia based on the famous play and novels by rose franken Brought to you transcribed Monday through Friday by your friendly neighbor who bottles coca-cola Relax and while you're listening refresh yourself have a coke and now Claudia Mr. Colligan, I didn't recognize your voice. You seem to have a cold. Oh I see it's just sign it's up No, mr. Killian isn't back from Chicago yet. No, mr. Norton isn't here either Now he hasn't got nothing wrong with him. He's just married and healthy and gonna have a baby That's the life Sure, maybe I'll try that one of these days, too You're the limit mr. Colligan Okie-dokie no message then Look if it's anything important I can reach mr. Norton He's only up in his country place working on the freight terminal plan That's why I don't like to disturb him unless it's absolutely necessary Okay, I'll just tell him that the cement contract is okay. Yeah on our terms Nice to talk to you mr. Colligan Sure, okay Okie-dokie goodbye now Lottie, what would you do if okay and okie-dokie hadn't been invented in the English slang? Mr.  You don't need it. Lottie. You don't need it I'd like to talk better for mr. Killian's sake. I mean anything less than perfect isn't good enough for him Lottie here's a little secret Mr. Killian's devoted to you okie-dokies and all Gee I'm certainly glad to hear that because I'm certainly crazy about him too. He scares me pink Because of his manners you think he disapproves of you, but underneath he sort of disapproves of himself That's the way I dope it out. You've doped it out as well as you handle that old battle axe Colligan I told him you wouldn't be in for another week or so What are you doing in New York? I brought Mrs. Norton down in the car The doctor wanted to see her for a final checkup. Oh gee. How is she? Okay She'll be along in a minute.  Gee you look great, mr. Norton all sunburned Well, I took a couple of hours off from work yesterday and went trout fishing. How's it going fine? Fine they're running fine. I mean the freight terminal not the trout.  Norton. That's another thing wrong with mr. Kinn He don't know how to joke with people and he's all tied up like in a knot You sort of feel sorry for him Say I hope there's nothing wrong with his son He's sure crazy about that boy of his even though we don't show it What do you mean Lottie about there being something wrong with his son? Oh, I'm not saying there is they didn't say there was either. They just wanted to reach mr. Kinnian Who wanted to reach him the school the headmaster of the school up in Massachusetts someplace I said mr. Kinnian was in Chicago. So he said he'd phone Mrs. Kinnian. Well, was that all he said that's all I guess it's just measles or chicken pox. They're always having epidemics in those prep schools Possibly I think I'll telephone Mrs. Kinnian to find out if anything's wrong.  You're a contradiction Lottie me Contradiction is that bad or good to me? I'm married to one. So I think it's good. Hello, darling Did you get your shopping done? Doors are jammed. So I said oh for heaven's sakes easier to do without and cheaper to Hello, Lottie. Hello, Mrs. Norton. Gee you look fine. He both do. What do you mean? We both do Mr. Norton's got a sunburn better than it. We get at the bar for that. He does look wonderful The country is just gorgeous this time of year. I can't wait till we get back. Can you how long will you be David? I just have to go over a little mail. Well, there's nothing important I haven't attended to or sent up to Eastbrook. Well, I want to do a little phoning If you don't mind my using my office for a few moments like go right ahead go right ahead Make yourself at home. Oh, thanks. Come on darling. Come on with me a man.  See you later. Okay So for Lorne David with the blind stone. Well, that's soon remedied. It's like it missed you anyway Everything all right since you've been gone Fine fine. Lottie's wonderful. Isn't she? I'm lucky. She's not beautiful, too I mean she's nice-looking, but but not dangerous with that voice Yes, she is She is what? What you said David you weren't listening to a single thing. I said you're just staring down at your desk No, I'm sorry. What did you say? I? Said I thought you were in love with Lottie Well, since you found out yes, I cannot live without. Oh, that's no joke Darling, what's the matter nothing? Why? Look like something's bothering you heavens help me if I really did have a secret to keep from you You've got eyes like an x-ray. It isn't mommy's it. I mean everything's all right up in the country Of course, silly. We only left a couple of hours ago.   I know I try to get over it Maybe after my baby gets here, I'll grow up who's to say what's grown up Maybe one of these days Psychologists will change the tune and come to the conclusion that more love and not less love is what's needed in this sorry old world of ours Then I'd be right in style Come in. It's only me Jeffrey Killian's outside. That's a relief Roger's son. Yes, I'm glad to say it is Roger's son He was what was wearing me Send him in Lottie. He doesn't want to see you He just asked for his father and when I said he was in Chicago. He said never mind He'd leave him a note. That's what he's doing now writing a note to his father Gee mr. Norton that kid looks awful and it ain't measles He asked me the school telephone because he's in some sort of trouble nonsense Lottie It's the end of the exam period all the old kids look tired out Maybe but it ain't that kind of tiredness He's right around the guilds and his mouth looks all tight Looks like he's lost weight too.  Just see that he doesn't leave now I'll be there in a minute and no matter what I say or do play along with What is all this about Jeff I don't know yet, but I think Lottie isn't too far from the truth Roger's been upset about him. I know that much He's made two or three trips back and forth to the school in the past few weeks I know The youngsters been in some sort of an emotional tangle Roger hated to leave for Chicago What about Mrs. Killian after all if Jeff isn't a jam of some sort he's got his mother to go to His mother's away. Oh Then the poor boys here alone Maybe we could take him up to the farm with us. That's exactly what I'd like to do darling But I have a feeling it's not going to be an easy thing to wangle So maybe you could manage him better alone David, you know sort of man-to-man Why don't I wait in Roger's office darling? You were a woman in a million. Thank you. Make yourself scarce. Hey, wait a minute come back Well, we put in mothers in the guest room Don't tell him it's the nursery it had hurt his pride Call me when you're ready. Can you pull up the blinds and Roger's room by yourself? Lottie take a letter will you Well, well Jeff I I forgot it was the end of the term come in come in great to see you I was just leaving sir.  Oh, no bother Well, well, you've put on a couple of good inches since I saw you at Easter. You must be easy Six foot, huh five foot eleven and a half sir. That's near enough One in the office and have a chat Jeff Mrs. Norton's using your father's room to do some telephone. No really sir. I your father. I'll be back at the end of the week Hope it isn't too hot in Chicago Here my office is a lot cooler. I don't know how Lottie stands the heat in this cubby hole of hers Now Jeff have a seat over there with the dad's Cigarette. No, thank you, sir home for good Schools over early this year, isn't it? It isn't over for another ten days Now look here as long as your father's out of town. I believe your mother's away, too, isn't she? Yes, she's away. I Went to the hotel She's away. Well, that's our good luck Since you're all alone in town Why not drive back to the country with claudie and me this evening very kind of you sir, but that would be impossible Oh, you're going back to school. I've been kicked out of school. Mr. Norton because expelled Kicked out describes it better. I Was kicked out of school once why?  He was a pretty good scout about it Father's are you know I won't be he detests a cheat my mother does, too My parents are very important people They won't like to have a cheat for a son come now. That's that's pretty harsh words what I am I cheat in the exams. They kicked me out Did you ever cheat exams mr. Norton Jeff? I never had to I had to I Couldn't help it. I see You don't see mr. Norton, but I think I do Jeff Yes, dear Well, Mrs. Norton I was just leaving you. Where were you going Jeff? To when list You're not quite old enough Jeff. I can lie about that too Jeff I think it's wonderful when a boy wants to enlist I hope when I have a son He's going to have that kind of courage But with your father away don't do it at least not till you've talked it over with him This is different Mrs. Norton. I've got to well, maybe but I Wish you'd do us a favor first It's a nerve to ask you, but if you could I what favor well We're so behind an outdoor work on the farm No help to speak of and David's been tied up so on those freight terminal plans But I wouldn't give for six foot of brawn and muscle to do a little mowing and raking You know for nerve of me isn't it David, but when I saw David stat Jeff Stanley here so big and strong It just seemed like a gift from heaven. It's no gift from heaven. Mrs. Norton your husband wouldn't want me on the place Yes, Lottie I did and if mr. And Mrs. Killian telephones tell them we've kidnapped Jeffrey He's going to pitch in and help us get some work done on the farm Okay, mr. Norton In the midst of a day shopping It's nice to pause at the friendly co-cooler for ice-cold refreshment And that's easy to do these days Because you find Coca-Cola chilled and ready for you in department stores grocery stores service stations around the corner from anywhere While you're enjoying an ice-cold drink remember to take a carton home, too The six-bottle carton is only a quarter which you'll agree is mighty little to pay for so much delicious refreshment Hi there, mr. King. Oh Lottie. Hello there. Hmm. I haven't seen you in a long time I've been having my nose glued to that typewriter machine Well, what do you think of it the typewriter? No young Jeffrey Killian. Well, he does seem to be in quite a mess Yeah, quite I think mr.  It means that I got a feeling that it's not going to be any cinch having that kid around the house Well, we'll see you'll see all right by tomorrow. Mr. And Mrs. Norton will find they're trying to chew more than they should have bit off Thanks for the warning. I'll pass it on to claudy and david So long lottie And as I was saying every day monday through friday Claudia comes to you transcribed with the best wishes of your friendly neighbor who bottles coca-cola So listen again tomorrow at the same time And now this is joe king saying out of war and remember whoever you are whatever you do Wherever you may be when you think of refreshment think of coca-cola Or coca-cola makes any pause the pause that refreshes and ice cold coca-cola is everywhere These broadcasts are adapted for radio by mania star and the entire production is supervised and directed by william brown maloney And now here's a word from your friendly neighbor who bottles coca-cola


Hey, what's going on guys? Root of the Nell here coming back at you with another Python tutorial. Let's get idle fired up here and let's see what we can do. We need to create a new window. Save this as, let's see, file.python. We can get a shebangline started up. Begin a class, base, define a constructor. Passing the self keyword as always. Let's create if the name is equal to the main. We can create a new object, root equals base, and now we can get started. Let's create a new string variable. Self.string equals, this is a string. It is, isn't it? And that can be a string variable. So now the function that I want to talk to you guys about today is something called rfind. And this is a lot like that function we were looking at in an earlier video called find. And what it does is it will find the current position, or at least the first position, of a substring that you pass into your function. So rfind does this in the exact opposite way. It will start looking at the, from the end of the string and going down. So rather than looking for, if we type this out, let's do print self.string and then we can do find. And if we, if we passed in is, it's going to start looking t, h, i, s. Oh, it's going to find is at that second position right here. It would return that, but if we supplied rfind, what it does, it'll start looking from the end. So we have a question mark, ti, spaces, and then, oh, there's an is right there. So it would return whatever position is right there rather than something over here. So we can try and recreate this all on our own. If we did that same thing and we printed out a function that we created that does this. So let's try and, let's try and get started. Let's do define, and then I'm going to call mine right find, because that's all it really is. I'm going to pass itself as always. We need the string variable. Substring is what we're looking for. And then we can have start, which is going to be zero, because remember we had some optional arguments in the find function. And that goes the same way for the rfind function, too. We have a start, and we have an end. End equals none, and we can create a code block here. So when we did last time, we tested if end is equal to none, if we haven't supplied any argument for that, we're just going to set end to be equal to the length of the string. And now we need the substring length, too. So let's do substring length. We can set this to a new variable. It's going to be the length function, or the len function. We'll just pass in substring, or what we're looking for. And position is going to be a variable where we save the current position. Since we haven't found anything, we're going to set it up to be negative one. Just remember, if we ran find, and it's something that we didn't find, like a z, it'll return negative one. So we're going to set that up for position for now. And that's subject to change while we're looping through it. So let's try it out with is. We'll at least put that back in there, and we can get back to our function here. What we're going to do is we're going to loop through it, as always. We're going to count with numbers, too. So we're going to use for i in range. Then we can go from start to end. Get our code block started. And now we can test if the string, and from the current position, we can slice on here. Then we can do i plus substring length. So the length of what we're looking for. And if that equals the substring, if we found the string, then we can do position equals i. So now it'll keep resetting the position for every time it finds something. And if once we get to the end and it's found this last one, what we're going to want to do is anyway return the position. So when we're done looping, we'll have the last position that we found. So it's going through all this, but it's going to retrieve this one here because it's the last one that we found. The effect of going through this way, though, and it finds this one first off by looping through all of this at once. So let's try it out here. Let's go f5, and we get 25 because we're running the original function here. So let's chain this to, at least let's put an R function here. Let's do print, and I'm going to put mine at the bottom. Print self.rightfind, and then we can pass in self.string. We're going to look for is, and that's all for now. We get 25 and 25 because we found 25, or at least the last is, is that position right here, and that's going to be the 25th character, that 25th position. So if we pass in another argument so we know where to start from, we can just say like 3, actually 4, that works for me. We can pass it in an R function too. We can set up 4, and this is still going to return 25 because we're only counting at the fourth position. So we're still going all the way through here. But if we do maybe 26, what we're going to do is get a negative 1 because we're starting right above this is. We haven't found it. So what if we did it at maybe 1, or 2, that works. We can do 2 again, and we'd still return 25 because it's the last one. But if we did, let's say the last one can be 20. Yeah, just play no 20. 2, and then the N can be 20. It's going to return 5 because it's going through position 2, and then it's going all the way up until it gets to 20. So now this would be the 21st, this I right here, that second is, so if we go from all the way up to here, the next is that we find is that position 5 right here. So that's why we return 5 because we're going through this way. But it's going to return the last one that we find, so it looks like we're working from the end. And that's how it works. This small function here, it's just the same as the find function, but with a little bit of modification, you're understanding this position variable, and that's what's constantly getting updated every time we find something, and when we're done looping through the string, we can return what we found, or the position of the last string that we found. So that's how it goes. Thank you for watching, guys. I hope you enjoyed this. I know this one was a little bit simple, and it's very similar to what we were doing beforehand, but it's just a new idea of being able to store what we just recently found in a variable and then return it when we're done looping. So yeah, it'd be nice if you guys could like the video, maybe leave me a comment, tell me what you think, and maybe subscribe. I don't know, it's just an idea. But I'll see you guys again in the next tutorial. Goodbye.


Alright man storms are brewing I can feel it. It's not that I can feel it It kind of showed me a storms of ruin the ladies and gentlemen welcome back to the channel My name is Jack the bus and we're back in Vinland saga, baby last episode ending with an apparent plot to kill Thor's this guy ask a lot of what he's been approached to be hired to I don't know if it's like because he Contemplated not coming or because of like him abandoning sort of I guess inherently his post all those years ago But it kind of threw it at us at the end if you're new to channel make sure you drop a sub and let's get started Troll oh Good, I'm glad they're throwing it right back into this actually Why Oh the troll of John oh troll It's a hero to you guys. Yeah, man. It's a yam like yams Vikings. He broke those cuz he left What it's been 15.    He's the most gung-ho I Sell return if he does come back. He's gonna be a changed man molded by war Oh Finish what I was saying when I return. Hmm. Okay, it has distinguished Ari from the rest of the group and of course Thor's and Thorfinn Oh my gosh Everybody confessing their love to Ilva This is so sad dude all these people are gonna be targeted because of Thor's too I know it's not his fault, but my gosh I wonder if Thor's has any inkling that he's good by going to be punished or they're trying to punish him for what he did oh Another love interest, but he's already got his eyes on Ilva Stop by Norway He doesn't want them to go to war because he he very obviously does not like it So he doesn't want that young generation to be involved Dang, so Ari and them all be safe W Thor's yeah for real Yeah, he's is he taking the men like his age too. No, he's actually going alone Oh my the question is will these men find Thor's before after he drops them off? Oh, man He's got this giant or Thorfinn's Where's Thorfinn? Oh, no wait, how small is Thorfinn? Could he have snuck on? There's not many hiding spots It's a pretty open boat dude the gigantic or he's rowing that side tire side by himself Holy giga Chad where's Thorfinn until I see him on land I will not believe that he's actually down on the boat. Maybe he could be in like a chest or something They have some supplies Dude, I would get so see I already get seasick on boats. Oh my god I can't imagine man Ward Magni Hakon Grimm always the only one I really Remember the name of from this group of guys, but we haven't really got to know any of them now knowing Thor's is a target I mean, I won't be that nervous unless those men find them like early on dude, he's out on the open ocean Look at that smile. Oh man, Thor's has to be freaking out.  Yeah the execution order.  That's how I knew he had bargaining power My question is how soon will they get there? How soon will they find them dude? I cannot imagine sailing like this like seeing nothing for like as far as the eye can see and all you have these ships I bro, I couldn't do it That's what I'm saying man falls the moon mayhaps they're really focusing on Ari Is he gonna give Ari the same lesson I taught Thor fan My son He just knows As are like the rest of the village apparently Okay, this is warrior Thor's though Give me a boy next day He's so different. He was To Norway isn't that where that first scene was where he had that like switch flip in his head When he wouldn't name his daughter That's when it started his percent is hers perspective shift He had a daughter to care for The pharaoh islands I like getting more of that from Thor's though like understanding why he is the way is hello Why he sees life like that? So this is like the only place most likely to stop for a while He said he was playing and leaving them in Norway The the drop goes hard and how do you convince everyone to stay behind? Oh, how does he know something's wrong It's quiet too quiet. Oh, no, are they here already they get jumped from up there. Oh No, are those the guys? Oh my gosh, there's a good drop all that on him. Oh My face insane, dude, where's Thor fan? Oh, yeah, he goes right for Thor fan trapped him in Well, who's through the last chair? I'm surprised. They don't just rain arrows down on them Well, I guess the target is just thorns Coming on ships. Oh my gosh. Oh Dude, this is an insane plan though. Like how long are they waiting? Is it pirates? They're on Viking ships. No, or can pirates be Vikings at the same time If they're experienced you can tell from this plan it's insane Horse knows it's gotta get real. What if he knows deep down like why this is happening? Oh shoot He's arming up gotta defend his son Only to protect I remember whenever he told him like you shouldn't have any enemies like this This is an enemy man.   I don't think he wants to use lethal force if he can can stand to he's just knocking him out dude That's what hasn't he's not like faltering or anything. He knows he was sent after like a strong warrior Oh Shoot it's going down man. I thought it was gonna be like an overarching plot or scheme like oh He's given him this order and it's like the first season is gonna be like about them like tracking Thor's down Nope So three we got the order the end of the last one found them 13 14 minutes into this one But I think this is gonna be Thor fence first like very big lesson about like not having to hurt people like he might have To come to grips with the fact that there are there will be people out there that will like try to harm you But again, I hope it explores that because like I said There are some problems with the prospect of not having an enemies because some people will want to hurt you in life Especially in war like they're going to but I do also like the sort of sea that's been planted about Flokey possibly having a personal grudge and this isn't just like based on orders And like I said, that's why I asked why I was able to sort of barter because then it might be an actual personal grudge It would make more sense like why would they why wouldn't hire up like hold a 15 year grudge for someone that abandoned? Maybe if it happened like last year sure like go find them But bro a decade and a half for someone that was like one of your strongest warriors that probably did a lot of work for you I'm not saying you owe it to Thor's but like you kind of do so it makes even more sense that this might be a Personal grudge or that fluky just straight up hates Thor's and why he wasn't able to show his hand in the village and just straight up killed Thor's right then and there because then everybody would know who did it and if he hires someone then how do you trace that back if ask Well, I'd keeps his mouth shut, but I next episode it's going down, baby Thank you for joining me for my reaction. I hope you enjoyed drop a like on this one. I'll see you in the next one Please peace