It's DK and I'm back. And guess what? Today was absolutely wack. Why was it wack? We made $0 a big zero. I'm trying to be a hero, but I couldn't. It was critical. We put in for some roadie orders. We didn't get them. We put some bs in from a carry company. We didn't get them. It just is what it is today. Some days you're gonna, it's gonna be like this. It comes with the territory.  Then you're gonna have some days like this. It just is what it is. There's no orders. It's a day after Christmas.  Well, New Year's Eve gonna come and then New Year's Day is gonna come. So everything is going to be still for probably, I would say the next two weeks. Come up below and let me know how it's going in your area. I do have some things I want to address and tell you guys that it is going to be some changes we're making to everything that we're doing. We are trying to add like two or three more carry companies. I have been in contact with them. I talked to them. We got a Swiss insurance and all that stuff and do that thing. And then we maybe be good to go with those carry companies. I'm not telling you guys to carry company. I do your own research.  And my business partner is going to read it. I'm going to tell you guys what I think about it. Okay, so your comment is from at BSG and they said your video is, you know, way off than what is actual than what it is in reality. They don't want you to give false hope to a lot of people because they buy a cargo van because of false hope and then they don't get loads or they get loads here and there. And that's not reality.  But the reason why I'm just going to say this right, we're going to give a disclaimer. The reason why she had to say it like that was because the words are I think they're from maybe from another country or some where English is kind of broken. So we had to she kind of kind of sum it up that way. So it is what it is. No, no, no, no slack to them or anything just is what it is. Now, I want to say this. I think that first off that you probably are new to the channel and you haven't been watching this whole time, because if you have been watching it, you would know that I do make videos like this where I only made $30 or $50 or a little bit of money and that one day. So I'm not giving anyone any false hope. I'm being very transparent with you guys and you low key. I'm going to say this. You make us. Don't make assumptions about things without doing your research. Like J-Main say, do your homework. You got to do your homework, which leads me into you not doing your homework. For those of you who out there who went and bought a cargo van or a Sprinter van and got this vehicle to do this gig working as well as use a carrier company and you failed. That is no one's fault but yourself. Everything needs to be blamed on you. You can't blame me. You can't blame big CJ. You can't blame the gig easter and whoever else. Pat entertain. You can't blame nobody. This is all your fault and you need to be a man or a woman about whatever you did. It is what it is. You can't blame somebody else for you not succeeding. The reason why you're probably not succeeding is because you didn't do your homework.   You something you saw on YouTube and then you just start to do whatever you was doing, which is probably wrong. And not to mention I have gave you guys a lot of lot of information. So maybe if you watch more videos, you will find out a way that you can do go out there and get busy. I have a video with 30 apps. Who else done it? I have a video with 30 apps. And if you don't, I mean some of them apps have to be have to work in your area. Some of them do. I don't know if it's going to be good or bad, but some of them do. You haven't done your research. Now, as far as me being a paid promoter for these apps, that is absolutely incorrect. I want to tell you guys this. I have put links in the description for different apps. And if you sign up, sometimes you get a referral, but that's it. And nobody ever called me and said, yeah, we want you to promote our app and we're going to pay this so and so and so and so absolutely not. I've never done it before. So I don't know where you get that idea from. I'm not getting paid to promote anything. You do get to click the referral call. I will be compensated for that. But I'll tell you guys that in the videos. Now, honestly, some people are going to win and some people are not. It just comes with the territory. So it's going to be it's a yin and yang to everything is going to be a positive and negative to everything. You cannot see here and blame anyone else for your failures. It just it just it's kind of sad because you're not taking accountability for the things that you did. Anything that happened in your life is your fault. I don't care what you say.   It's not the teacher for you didn't do your homework. You're into your research. The teacher gave you the information. You was supposed to study for the test. You didn't study for it. You just went out there took the test and you fail. Now you want to blame the teacher. That's not how life works. Take accountability for the things you are doing. You want to need to in order for you to see you're going to do have to do a lot of research. You have to be dedicated. You can't be lazy. It's not going to work if you lazy. If you lazy, you want to get up in the morning. You have to have discipline. You have to have drive. And I told you guys this multiple times before. You need to have money saved up before you get in this business because if you don't, you're going to make bad choices. It's not going to work well for you.  So you got to hurry up and try to do this order. It's terrible order. You can fit this in per mile and you actually losing money instead of making money. So I advise you or whoever else they may say this about the youtubers or the influence or whatever you may call them. Stop blaming everybody else for your problem. This is your fault. It's not nobody else fault. You didn't do what you were supposed to do. Now you are going through a terrible time and there ain't nobody else fault with your own. So it is what it is. I understand why some people may think that I'm not giving nobody. I ain't selling no hope. I never told no hope to nobody. I never said this. I never said anything that was incorrect to anybody. I'm just telling you guys what I do and that's it. That's all. I need you guys to comment below and let me know what you think about this comment. As a matter of fact, I want y'all to say this. How many people have helped succeed? How many of y'all have gotten information gems and made money off the information that I've given you? Comment below and let me know.  I'm not selling anyone false hope. I'm probably one of the most transparent gig tubers out there honestly. If you disagree, comment below and let me know. Tell me I'm lying. Tell me I'm lying. But that's what I feel about the situation. I understand you know everybody can think how they want to think. But again, it's not just about cargo van business. It's not just about the gig. It's just about life in general. Be responsible for your own actions. That was one of the things that we used to say in grade school or after we did the Pleasure of Allegiance we had to say um what was it? A few moments later. This is what it was. I am somebody who believe in myself. I'm responsible for my own actions and no one else. I will solve problems peacefully. Education is my number one priority. I'm a Neil Shining Star and I will go far and there's something else about that. But that's what I did. I learned this in grade school. They told us that it's a grade school. You are responsible for your own actions and no one else. So be responsible. Do your research. If you don't do no research do not start. If you don't got enough money save up do not start. If you don't have dedication do not start. If you don't have drive do not start. Just don't do it. It's not for everybody. Everything is for everybody. So I appreciate the uh the comment but hey look in the mirror. You gotta blame yourself. You cannot blame anybody else for anything that happens with you. It's all on you. That's all I want to say. It's time for us to clap and hey babe. I hope you guys enjoyed this video. I'm sorry that I could not go out there and get busy. It just is what it is baby. It comes to territory. But make sure you guys lock in tomorrow. It's going back down. We ain't playing around. The 2023 is still big bags. We'll keep on giving a reason. I'll see you guys on next one. We're going, we're going, we're flowing. You already know the DDK and I'm on my way.


What advice you would have for organizations? And also you said, you know, myopic view, because in a security space, it's very popular. Night lamp, you know, you're looking where you expect problem will happen. But as you said, you know, the good guys, as I say, the bad guys have to be right only once. And as a good guy, you have to be right all the time. That's the big difference. So talk about what advice you have for organizations that not only they, once they get the pin, the target on the back, they will be attacked again and again and again. But is there something that they can do culturally or practice-wise that will at least offer them a greater level of protection? And that's why I like to talk about the cyber kill chain.  If you stop any one of those, you're gonna break the chain of attack. And so I'd like to think they have to get it right multiple times to be successful and we have opportunities to stop them. And so how do we do that? You know, the first is understand your attack surface. You know, make sure that you minimize your attack surface and then be looking both North, South and East West. You know, have internal segmentation and visibility internally. So as they try to spread, you detect that. Next is, you know, update your playbooks. Make sure they're validated through exercises. You know, it's incredible that you go through if it's a tabletop, if it's a technical exercise, you get leadership expectations right. You know who's involved. Those playbooks have to be validated. And almost quarterly would be, you know, if you have a quarterly exercise, you'll get through all your critical playbooks. You know, next, we have these indicators of compromise. You know, it might be coming through DNS. It might be coming through, you know, warped traffic. You need to really kind of do a gap assessment and say, how am I able to monitor outbound traffic? How am I able to monitor internal data flows to discover indicators of compromise? The next is kind of a different one. You know, it's the traditional cyber hygiene.  Are you doing security training? Those basic things that we should be doing. And finally, outside of the security team, work with your legal team, because we're seeing more and more laws come out that says, if you live in this state or if you live in this region, it is illegal to pay a ransomware or you have to notify if you do. Or, you know, and so make sure your legal team is plugged into the regulations in that environment so you don't do something inadvertently that's not within the law.


Hello, do you know what happens if you put 12 ad breaks into one YouTube video? And how do you even put a mid-roll ad break into a YouTube video? I'm going to show you what happens when I ram one YouTube video with loads of ad breaks. It's a two-part video, this is part one, I'll put what happens next as a link in the description.  Have you seen what's on YouTube lately? We need your ideas, we need them now. So let's dive now into my YouTube studio dashboard where we can see which video is going to get the ad break treatment. Now I'm going to scroll down to this section here on my analytics, your top videos in this period. This is the reason I'm doing this, my channel is really broken. I want to niche my channel so that's partly why I'm doing this crazy experiment. If you look at my top five videos, one, two, three, four, five, there, look at this one, my valent boiler remote control manual. This video has me reading out the instruction manual from beginning to end for 40 minutes. This was in April and it was from the time when I was trying to do lots of experiments to get 1000 subscribers. Well I did that. Now I have a channel that's just full of stuff like this. Now I've got a problem because when someone sees this video, they're not going to want to watch all my videos about how to get ideas out of your head and onto YouTube. They've got their information about the central heating. They're not coming back and that I think damages my channel in the YouTube algorithm. But the problem I've got is look, it's making me money. It's maybe $20 so far. Now I'm trying to take a long term view on this. One option is to just delete the video but I don't want to do that because it's earning money and it's bringing in views but the views are the wrong kind of views. The video lasts 40 minutes so what if I put a lot of ad breaks in it? What would happen? Would it reduce the overall watch time or even the number of views or would it make the revenue go up? So that's basically the experiment because I need money but also because I need to start to move these videos off my channel. So I might as well play around with them first. So I don't recommend you do this. I think I can be the full guy for this particular experiment. Now the way to add an ad break, firstly obviously you need to be monetized but I think you already know that. You need a thousand subscribers and 4,000 watch hours in the last year. You can see that I have monetization on and I have ticked everything I could tick. Down here on the monetization tab because the video is longer than eight minutes there's a section here called manage mid rolls. I've got a few mid rolls but thankfully this shot cuts them out and leaves them out of shot. But with these mid rolls, look I've already have five ad breaks in here. The first one's at 11 minutes 57 and you can see on the top left hand corner these are the other times of the videos. I think these are the times that YouTube placed automatically and I like this really sneaky one at the end at 38 minutes 54 seconds which is about 10 seconds before the end. If you don't want five ad breaks in this video I'm going to double it and add two more make it 12 ad breaks. So firstly let's remove the ad breaks which are here and it's really easy to remove an ad break.  We call them rubbish bins here and I'll just delete, delete, delete. So I wanted to place the first ad break at five minutes and the way I would do that is if I run this cursor up to five minutes let's see what's happening around here to place our first ad break. Outside a set time period. Let's just put it there. I don't care. I'm going crazy with the ad breaks today. It's very easy. You just hit add ad break and there you go. There's a nice blue line to indicate my first ad break. If you want to save at any point you just hit continue here and that's it. It's placed the ad break but we're not going to stop there. Let me take you into my analytics and you'll see why I've chosen three minutes. My average view duration, which isn't easy to say, is three minutes and two seconds. So I thought why don't I just make the ad breaks every three minutes. Hit let's go with the mid rolls. I've got the first one at five minutes. Let's put the next one at eight minutes and I guess we'll find out what that is. The cliffhanger. I like it. Add ad break. The next one I'm going to do at 12 minutes. Hey, I could have a little counter in the corner adding up the ad breaks. Regarding the active time period, add ad break. If you're watching the actual video I'm putting these ad breaks on I'm so sorry but you know my daughter needs shoes. Let's put the next one at 15 setback mode. That's a good one. Ad break. 2200 hours ad break. Next one will be at 21 minutes back temperature ad break 24 minutes and can no longer operate the controller. Oh, that's an ad break. 27 minutes in. You better believe that's a mid roll ad break. 30 minutes without taking time periods into consideration ad break 33 minutes ad break. The following service messages may appear 36 minutes all together now ad break. And finally 39 minutes into this 39 minute video. Ad break. Now I make that 12 ad breaks but before I hit save I'm going to try one more experiment. I'm going to try and get one more ad break in and I've always wanted to do this. If you go right to the very, very, very beginning back to zero colon 00 will YouTube allow me to add a mid roll ad break at zero seconds into my YouTube video. I've got 0000 in the time code so you can type it straight in there and we'll go up to add ad break to see what happens. Yeah, it's done it. Now I don't know if that's going to work because technically this is my first mid roll ad at zero seconds. So I'm kind of adding an extra commercial onto the ads before my video starts. Who knows what effect that's going to have on my watch time for this. I don't care because I don't want this video doing more damage by bringing me more viewers. It's a bit like the plot of the movie The Producers, isn't it? I want that money! I fell on my keys. I'm always trying to make a profit by making people not come to see my show.  So I'm going to hit continue here and it seems to have accepted it. I now have a YouTube video with 13 mid roll ad breaks. If I hit the manage mid rolls button, there they are. 13 of them. Will it be money or reduced watch time? What do you think? Leave me a comment below. Make your prediction now before I release part two of this video so you can see the actual impact that this has on my video and my channel. Our 13 YouTube mid roll ad breaks are now locked in and they can now do their magic and spread their news of wicks and honey and Grammarly. But let's take a look now at some of my other long videos that I think are damaging my ability to niche my channel and to have my channel be more about getting ideas out of your head onto YouTube. So I'm going to go now to how to make a bus blind display case. Now, I love this video because it's 14 minutes long, but it does not connect with any of my other videos on the channel, except maybe my bus videos, which again, I've got to move. So let's get to the mid rolls. Now, at the moment, I've been very restrained because I have one ad break at 1226 in a 14 minute video. So let's see what happens if we add the ad breaks and we'll add ad break right at the start at that zero, zero, zero. That's our first ad break. I'll put these every three minutes as well. I'm just going to put them straight in without really checking if they interrupt anything. So ad break at three minutes. Let's go to six minutes. I could just type it in here, couldn't I? Nine minutes ad, ad break, 12 minutes ad, ad break. And we can move this final one. You can actually drag them along the timeline as well. So if I highlight it, you can see it turns blue and I can just drag that ad break over to say 13 minutes just before it ends. So we now have six ad breaks instead of one. Link in the description takes you to the update video where we'll find out what the effect on my revenue is for this video. For context, my bus blind video earned me 44 cents this month. We'll see if that figure goes up. What I'm going to do now is to find my top viewed videos in the last 28 days and see if there's a long one here that I can ram with ad breaks. Oh, this is a good one. This is how to buy postage in the UK. It's an absolutely gripping video about how to buy stamps online. This video has already earned me $27 from YouTube. Let's see if I can get more money or if the views will go down. And this is one of my best performing videos at the moment. There's only one ad. And I think that was placed automatically by YouTube. So how do we do this? Let's put one at zero, zero, and I'll put one at three minutes. Five, 45, ad, ad break. We've got one at the end. Let's put one in just before the video finishes. So the video finishes at 9.42. Let's put one in at 9.41. So we now have five ad breaks instead of one. Don't forget to hit save. And just as a control experiment, I have three videos about me traveling on Southwestern Railway. And I'm going to add an ad break at zero seconds on each of these to see what effect that has on the views or the revenue. That video earns me 31 cents per month. That's the figure we're looking to beat. Now I need to put a warning on this experiment, which is, I don't care what effect it has on my revenue or my watch time for those videos, because I think they are being quite harmful to my channel anyway through their success. But I have no guarantee that YouTube will actually serve 13 ad breaks in that first video or any of the others, even though I have put a mid-roll ad break at zero seconds. I don't know if YouTube are even going to play along with that and actually supply a video at zero seconds. But who knows, I'll be watching the videos myself to see if it looks like an extra ad has been played. But I think the only way that I'm going to tell whether or not this works is to check the update video that I'm going to make in one month's time. That video is right here. I suggest we all go there now to see how much money I earned or to see how much my watch time or video views have plummeted. I might have crashed the channel. Here's what YouTube thinks you should be watching next. Thanks for coming along with me on this dangerous experiment. Now, do you know what would be really funny right now is if I put in an ad break.


This you want to remove it and I'll just do the clicking. Yeah. Yeah. Okay. All right Good afternoon everyone. My name is Carmelo Ragoza and my name is Tariq Eli. We both work at SAP SAP and primarily focus on bare metal as a service so Yeah we're going to present the work that we've been doing in in this context and We first we're going to kind of give a quick overview on how this fit in all SAP plan of moving into OpenStack And the presentation then will focus mostly on the bare metal as a service In terms of content audience We think that the type of work that we've been doing can be beneficial for It is more companies or enterprise that can share the same type of requirements So we're gonna go through quickly this quick background on SAP what we called called converged cloud plan And then we show the current in our solution that we've been we developed and we are using in production Then we focus on in the enterprise requirements that We are relevant for us We don't we go after in the OpenStack evaluation. So together what word our fundings in that and and then how we actually integrate the Hyronic into our bare metal solution and then we show some of the future work So currently in SAP there are lots of different clouds And so they are kind of silos They are self-contained the work on their own, but there is already a plan in place and we are moving Towards a single Solution So OpenStack is at the bottom of that to manage all the different type of hardware And and then we have all the various applications that we have That can be using the some pass a solution application that can call directly OpenStack and applications that can use some Hinaus layers platforms So our bare metal work fits into this because some of the Applications they need this type of solution So currently in SAP as I said we are moving into into this Solution which we call the converged cloud now Maybe some of you already heard about SAP HANA just quickly HANA is an in-memory database and Allows you to Loads hold it and customer data into the memory and from there you can do Analytics and this interesting stuff and HANA works with the mostly with the large Ram footprints so in terms in terms of like terabytes of RAM And so in order to have HANA working in a cloud setup We have two different Aspects that we were taking care of first one is the what we call the cloud cell HANA cloud cell and is from the Infrastructure point of view where we have standardized the different type of hardware and how it has to be Combined together. So we have one part which is the compute You can see it pretty straightforward. There are different Different switches You can have different type of networks or two for management and for the normal actual customer Networks and Those nodes are highly dense the in RAM Because as I said HANA normal deployments are in that order and then we have separately a storage rack where All the storage is kept So this is this is what we use to provision HANA landscapes and This is Allows us to have optimized performance scalability reliability and security The other aspects The other aspect that we have instead is the software that allows you to Deploy in this landscape infrastructure, sorry it's called the cloud frame manager and It basically allows you to Manage the life cycle of your HANA landscapes So you have a customer that need to have What we call that frame In this frame, which is made of one of more nodes. You need to deploy your HANA and Our in-house solution allows you to do that and it manages all the low-level resources so storage networking servers and so on I mean and this is great this works fine, but of course There are some kind of limitations in the long run and So when we started to think about moving forward We say, okay, let's look at what is if you can have something that is open. That's just in-house and so we started to think about in terms of what the kind of Requirements do we need if we want to go that direction and so the first one of course is the ability to Manage your bare metal infrastructure, so you need to be able to do that In terms of servers Network storage and so on so you need to have a solution that allows that we have it in-house But we want something that is open That is because And it links to the second bullet points if we want to bring him new vendors at the moment what we have to do is Writing a new software that allows you to do configuration for that new vendor The more they come in the more we want to use the more we have to do that We will we have to maintain and so on on the other end We don't want to be locked to just on a single vendor So we want to be able to detach from that and rely on something that kind of go on his own It's the vendor itself that writes that part and you just leverage on that and The other requirement was to have open API so open interfaces that are just accepted that they are used and so we can just Rely on those Another important aspect is a multi tenancy. So multi tenancy in In cloud you have the true VMs bare metal. It's kind of easy. You give a box just to a User, but there is also the multi tenance in terms of networking. So when you provision a Server you want to be able to use a Tenant network You don't want to you want those to be separated for each tenant that you have in your landscape So that was something that is really important for us that you achieve it Obviously by using things like VLAN VXL and so on so What we were looking needed to have that type of features We also needed to have network reliability. So we have reliability in terms of Servers in terms of storage, but we wanted also to have in terms of network. So what that means is that you have Multiple networks that are bonded together and that if one fails You still be able to carry on your provisioning and Finally, we wanted to be able to support the multiple deployment models. So you can have local Deployment local boot you can have NFS route and so on So these were the requirements that were important for for us So when we started to look on what we could use Of course OpenStack was one of the Most wanted candidate for that so in our evaluation we realized that out of the box We could already have things like the active support of multiple vendors Is there each vendor that come in the OpenStack community once is hardware Supported it does the development of whatever is it's a plug-in and so on so that is It kind of ticked the box there for us And we have standardized API of course and that's that's easy is accepted in OpenStack Is there and it works fine with us? Multiple deployment models in there. We we didn't have the local boot CFM our in-house solution does NFS route so we wanted to have we ended our plans to have the local boot and Ironic supported that out of the box. So it was a good thing for us To be able to use that feature right away and the other thing which it's It's very relevant. It's very important We wanted to have an active community because you can have the solution as best as possible But if you don't have support behind that You're gonna be there stuck many times and we found that The community in OpenStack is great in terms we've been Actively engaging in the community in terms of new features and bugs and so on and Just like I mean, there is just an example over there bug report in IPA Patch committed just within few hours, which is great. I mean if you think about it You find something that doesn't work talk with somebody that knows what he has to do and it just fix it and and also the last bullet is The ability to have ironic a standalone mode why that Because how are in our solution is already working is in production We cannot ride away bring all the functionalities that are there in OpenStack. We want to do it gradually And so the ability to just use a single component for that It was quite important for us because right now we have a solution that we've been integrating ironic that it's working It's out there so in In our evaluation we then come up with a number of findings, right? the right support We we needed to have Rating capabilities because what CFM does during the Provisioning it that configure the node with the exact rate support that that you need and we needed that so although at the time it wasn't there for ironic, but to we It was nice to to know that they already were working in that direction Multitenant networking so ironic up to now I've been only supporting flat networks. So there was no the possibility of having and using VLANs and Although Newton supported that that there was no integration between Newton and ironic But we've been working with the community and this that was a short come. We changed into a feature Talking about that later on One of the things for example right now is the at the moment ironic doesn't support NFS route It's something that we are kind of working with and perhaps in the future. We may contribute it And one of the things just a comment about the logging it can be a bit difficult at times you need to go through many Many files, but is the way it is is there It's just on the logging in other aspect of us that the we wanted to integrate and Open stack components with CFM it has its own logging components And we wanted to be able to if we trigger a provisioning workflow We wanted to make sure that we can integrate all the relevant to logging entries from the open stack component No, the CFM workflow so we can see if something goes wrong Which what happened in there are then going to look into the log files you can see at the CFM also So this this point refers to integration of the back-end open stack logging capabilities into the CFM logging capabilities So and as Kamal said to me log files, it could be difficult to actually go find out where the exact Issue is but we wanted to integrate that into the CFM logging so we can relate to a specific provisioning transaction that we triggered from the CFM And the last bullet is at the time of the Evaluation that we were doing there was no hardware discovery But these actually then come up as a nice another feature in in ironic and he's already there so Now how we've been going for the integration as I said because you didn't want to go and integrate right away all the open-stack Components we went for integrating only I ironic only in standalone mode the way we did it was by wrapping up the Ironic API calls and so that they could use the in a CFM as Incrously, that's because when CFM gets a request to create a frame He creates a kind of a plan and that plan. Yes, there are many different tasks so because we wanted to be able to create a task for Ironic we needed to have this kind of approach The other things instead like discovery networking and image provisioning We just kept the one that we had developed in CFM So the discovery was done through CFM CFM has a Discovery script so once you boot it This discovery script starts it's collected in node Information and then what we do we use that node information from CFM which invokes Ironic To create a node and using that type of information that has just collected then once you get a reply back with the UID We store that UID In CFM databases so that that can be used later on for any other operations Pretty simple. It's nothing fancy there The other part instead is when we do the deployment Once we kick off the deployment We use the Ironic Python agent to do the bare metal node configuration So we do like create configuration and writing the image to the disk And then after that in order to do the node customization we use cloud in it and CFM Dynamically generates the config drive and user data for cloud in it so As a future work, so this is what we've been doing so far and as I said This is something that is already implemented and integrated in our Solution and it's in production. It's used As a future work and We've been working with the community. I was saying that earlier about the Newton and Ironic integration and This is something that apparently many Companies or anyway open-sync users who wanted to have and Is the ability of when you do a provisioning of a node to use the tenant network? We'll be doing that to work in the community and We jointly with Arista And now this work is in mitaka and of course we need to integrate with Keystone for identification authorization and this is mostly because if you think about the Cloud converged cloud the plan that we showed at the beginning that work will go into the whole SAP and Infrastructure it means that we are going to use More open-stack components and so we will need to integrate with keystone which would be served in the in the context of the converged cloud The other thing that we've been working is about the The discovery so We had our own discovery, but we want to leverage the Ironic inspector one and And so we've been already doing some some internal work in order to integrate our own discovery and Ironic one in and using just a single RAM disk that can decide which one to use at provisioning time We also been looking at Nova in terms of the Scheduling availability zone affinity and affinity and so on this because we Need some how a way to have a bit more control on that Reason is that the HANA can be can have multiple nodes for example and depending on the customer Requirements you may need to have a rack specific that you want to push Specific HANA instances into with the same rack and so we've been looking into Nova to use some of that capabilities and finally the use of NFS route as I was saying we've been working on that to integrate it with Ironic and that could be done by decoupling the deployment and Boot booting operations at the moment when you do that in Ironic when you do a provisioning just a single operation If we are able to decouple that we can just do the boot from NFS route and then from there It just take care of itself and we have something that is Almost there and eventually we might decided to push that back to the community And I think Yeah, I can take questions if there are any questions. I think yeah need to use the Hi great great to see Ironic in action Could you please elaborate on the rate configuration and do you do it during the deployment itself or sometime in advance and Do you have some uniform rate configuration or it's various per instance? And so we do it in during the provisioning workflow. So that's where Our requirement was a bit different than Ironic because I want to do that does that in the cleaning phase because we want to do that during the provisioning phase and So in we have some developments ongoing where we are we're using the target rate config still to specify What rate information you want to do but we have a custom hardware manager in IPA that actually triggers that during the provisioning and Then we also wanted to have the capability so we can specify different types of rate depending on the workload because different workloads might require different types of rating and so we're still using and Some aspects of Ironic, but other than relying on the cleaning based rating. We're doing it in during the provisioning Thank you


Baroness Linda Chalka, many thanks for coming here to WPC-TV, you are a former Minister of State for Overseas Development in the British government and a big expert on Africa. You do a lot of development aid for Africa now, but I want to ask you a slightly different question. Is it a good idea for the 0.7 GDP target to be kept or to be aspired to by donor governments? And secondly, somewhat related, what do you think of the most recent band aid concert for Africa? I ask the question because some people think it's rather patronizing. Well, first of all, may I just correct you, for the last 18 years I've been committed to taking investment into Africa, trying to enhance trade. Which is different from development aid. It's very different from development aid. As far as 0.7% is concerned, this of course was a figure arrived at years ago when GDP was far lower, but Britain has, in the last few years, committed itself to paying 0.7% as a minimum and although times are hard, and I know many people disagree with this because other things have to be cut back. I think it is right to help people in their own countries because if people are helped to learn and trade in their own countries, they are less likely to try to be refugees in other countries. Also, there are so many people who come to Europe as refugees who would far rather have an opportunity in their own countries near their own families. And I hear that comment again and again. I'm afraid I was abroad when the row broke out about band aid. I think the important thing is that people see that money is being well spent in the developing world. And therefore, however the money is raised, as long as it's legally and properly raised, I don't think people have an objection. You only have to look at the enormous contributions to children in need in the United Kingdom or to the Red Cross or to a disaster. And I know from my own post-bac and my own involvement in a number of NGOs as an honorary patron and vice president and so on, generosity is really there for the developing world, which is why I think we're right to stick to the 0.7%. Just a very brief follow-up question. Investors, what are the biggest obstacles? Is it corruption? Is it border problems? Is it bad governance? What is... It's a combination. I mean, I ask the question because a lot of people would say, I think quite justifiably, that Africa really should be the continent of the 21st century. I believe Africa should be the continent of the 21st century. It's only going to achieve that if it improves both its commercial governance and its government governance, if I can put it that way. If it removes the barriers to investment, which is what I do through the investment climate network for Africa, that is a PPP, which takes account of the real needs to change processes of government. And that includes border processes, which are a great impediment to inter-African trade. Buenos Nido chocca. Many thanks. Thank you.


So, in previous videos, we've talked about how to draw regression curves. We started with linear regression, we talked about quadratic, polynomial regression, aka power functions. We've also talked recently about how we can do exponential regression, but what important question to ask is, what is the right regression to use when you have a data set? So, for example, look at the following data set on the screen. When you look at this data set, it does kind of look like it has some type of exponential decay. If you connect the dots, you might get something like this. So, it seems like exponential would be a good model, but maybe there's really not much curvature to it all. Maybe it's more of like a polynomial type thing. I mean, there could be curvature, but I mean, what if the polynomial behavior is something like this, right? Or maybe there's no curvature, like I said by mistake a moment ago. What if it's more like a line of best fit, you know, I mean, that doesn't seem to fit exactly too well. Again, exponential looks really, really close, but we'll see in a second whether that's a good fit or not. Another option is look at this data set right here. Again, you see this data, which actually there's some clustering that seems to be happening near the origin, right? So, if you try to draw that data, it kind of looks like there's some curvature to it. It's not exponential, but when you zoom out to the larger scale, again, all those things kind of bunched together, and again, it seems very natural that, I mean, you could try the exponential growth, right? That seems acceptable, but maybe it's like a linear growth, right? It can be sometimes difficult to tell which data are given a data set, what is the correct function family to model using. And so, let me further complicate the issue with the following picture, right? So what you see on the screen right now is an exponential function. This would be an exponential function, which is growing with a 1% growth rate over time, okay? And so, you look at that scale, it looks very much like an exponential function. If I zoom in to a scale like the following, let me turn this off, let me turn on a different function. This right here is a linear function, so this is y equals 0.01x plus 1, so this would be a linear function, which y is intercept is 1, it has a 1% growth over time, all right?  So the point, of course, is what happens when you look at these graphs together? When you look at them, you're like, well, did the colors changing, right? Because they look almost identical, right? Kind of like the Weasley twins were identical. You look at the two, you look at the two functions, you can hardly see a difference on them whatsoever. What happens is you have to kind of come out from a very far scale to start to see a difference, right? It wasn't until he came to like x equals 200 that it would start to be significant that these are not the same function. And then it's not really until you hit like x equals 400 that you really start to see that the upper function, clearly they're different, but it's not a linear function with a constant slope, it's an exponential function with this constant ratio of growth. And so, again, it can be very difficult to see on the small scale what is the difference between these functions. And after all, when you zoom back in, like I said, you can't really tell much of a difference between these functions. And as a mathematician who lived through the year 2020, that is when the COVID-19 pandemic was at its height, right? I think I have to say I was horrified to see so many online comments about people talking about the spread of the disease and not knowing the difference between a linear and exponential growth. I saw many people say like, oh, well, we don't have to worry about the coronavirus right now because it's only linear growth. But once it becomes exponential, then we have to worry about it. It's like, well, it doesn't just switch from linear growth to exponential growth. The spread of the virus was always exponential, which itself is not the correct model. I mean, a logistic model would be better in that regard. But that's a topic for another video, right? What they're really trying to say is when it switches to be exponential, we have to worry. What they're really saying is that exponential growth, if it has a small enough growth rate, looks a lot like linear growth, right? And so it's hard to tell the difference. But of course, if you allow the experiment to continue long enough, then you do see a stark difference between the exponential growth and the linear growth. And so when it becomes exponential, that's what people are talking about, although erroneously speaking. So let's go back to those data sets we saw before. So we had this data set which has some curvature to it, at least it looks like it's curvature. Maybe there's just some error there. So how do we tell the difference, whether it should be exponential growth or whether it should be some type of polynomial growth or power function growth? Well, if your function looks like y equals ca to the x, that would be like exponential growth. But if it looks like y equals c times x to the a, that'd be some type of monomial growth. Which one is the right answer? Well, hit both sides of the log. It doesn't matter which log you use. You use the common log, you use the natural log. It doesn't matter. If you do that with the exponential function, you'll get the log of the data should look like log of ca to the x, which if you pull it apart by logarithmic rules, you're going to end up with log a to the x plus some log of c, right? C is just a constant. Then you can pull the exponent out. So this becomes x times log of a plus log of c here, which I should mention that log of c, that c is a constant. So log of c is just a number. And then log of a, again, it's just a number. We can kind of be like, oh, we're going to call this m. We're going to call this b.  In other words, if you take an exponential function and you hit it with a log, it's going to look like a log, OK? Now, on the other hand, if you take the log of both sides of your power function over here, log, log like so, well, then the left hand side is going to become log of y. We're not going to really worry about that. The right hand side, you're going to get log of c x to the a, right? Which you break that apart, you're going to get log of x to the a plus log of c. We're going to call that part b again, right? It didn't seem to make much of a difference. But as for the exponent here, you're going to get a times log of x plus b. And so in this situation, your function is going to look like a logarithmic function, a log versus a line. And so that's going to give us something that we can distinguish between these two. So let's come back to this data set right here, right? If I take this data set right here, this looks like the data set x comma y. So what we want to do is we want to switch it so that we then get x comma log of y. What happens if we hit all of the y-coordinates with the common log? So that's what we're going to do right here. Oops, we don't want to see that thing. Notice what happens is if you take the logarithm of each of the y-coordinates, you get something that now looks like a line, right? And so that then agrees with our assumption earlier that, hey, it looked like an exponential function, right? If I get rid of it, it did look like an exponential function. But the point is when you send it back to take the log of all the data, you get a line. So that tells me, oh, I guess it was an exponential growth, right? Let's look at the other data set right here. So this one, again, the data was really spread apart, right? There was this big jump here. It kind of seems exponential, right? Especially when you zoom in here. It looks like there's curvature to it. But when we take the log of both sides, excuse me, take the log of the y-coordinates, you then get a data set, which it's like, it looks fairly flat in this region right here. But again, zoom in, right? Those data points are all bunched together. You can see there's curvature to it. And so really, if we were to put all of this data together, this would actually suggest some type of logarithmic growth right here, which means the original data set actually was, since it looks like a polynomial, the original data set actually looks like it was some type of power function growth. And so use logarithms to help you decide whether a data set should be exponential or whether it should be a power function.


So, today we are going to work on relational database design focusing on functional dependencies and normalization and also touching a little bit upon other kinds of dependencies, multivalued dependencies in particular we will touch upon briefly. We will also discuss the practical aspects of database design. We will cover the formal aspects through functional dependencies, but also some of the practical aspects will be touched upon later in the talk. So, the first part of today's session is fairly theoretical. Along the way I would be very happy to take questions. People usually find normalization somewhat confusing area and when I say people I include myself, it is fairly deep theory. Also people find it questionable area in the sense that if you do ER modeling properly, the question is why do you even need to go through all this normalization theory? Does it matter? Does it have any impact? And the answer to that question is partial yes. If you do your ER modeling perfectly, most likely you will not have very much to do later on when you do normalization. However, not everyone does ER modeling perfectly. There are likely to be glitches and this is what normalization theory will catch. Now, normalization theory started well before the ER modeling even showed up as a topic. ER modeling came quite late actually, quite late in the sense of maybe 7 or 8 years after the initial papers on the relational model. So, during this time and in fact later also it is not as if ER modeling caught on overnight and completely replaced everything else. It did catch on, but other approaches continued to be used. In particular, the theoreticians approach to this whole problem of normalization was to start with what is called the universal relation. That is think of all the attributes of all the things that we may want to model. Pretend there is a single mega relation containing everything, all the attributes. Of course, the attributes have to have unique names. So, that two different things do not have the same names. So, for example, if we think of courses and courses have an ID and we think of people, people have an ID, course ID and people ID should have different names. So, you start off with a mega relation where each attribute corresponds to some attribute of a real world entity could be physical, could be logical and there is one mega relation which is called the universal relation. And then you start from there and apply the principles of the theory to break it down into something which does not have problems. The first step today is to understand what are the problems that normalization theory even tries to solve. So, let us look at a few of these things. This may be a repetition for those of you who are already familiar with normalization theory, but I think it is good to refresh the minds of everybody before going deep into this theory. So, first of all, our ER schema, the university schema was developed using an ER model and then normalization theory was applied to make sure everything was ok. So, the designs that we have used so far are actually good designs. There is nothing really wrong with them. There are alternate designs which are also good much like when you write an SQL query, there are many ways of writing the query and you cannot always say that one way is better than the other. They are just alternative way, but there are also bad ways of doing things. And to show this, we are going to do something unnatural. We are going to take this schemas that we have and munch them around to create other schemas which illustrate what kind of problems can arise. Sometimes it is obvious, if you look at it, you would say why are you doing this? You are mixing together attributes of two different entities and that intuition is usually correct and it is formalized a little bit more using functional dependency theory. So, let us start by taking our instructor and department relation and for some reason if we combine the attributes into a single relation. So, what are the attributes there? If we combine instructor and department, we have for instructor ID name, salvi, department name and from department, we have department name building budget. Note that department name is the same thing in both the relations. It is not two different things. It is the name of the department. Therefore, it only appears one time in this particular combined schema. If you use the universal relation model, if you had instructors, you had departments, you just put together all the attributes. This is what you would get. These are the attributes of instructor and these are attributes of department. Now, what kind of problems arise? Supposing we assume that a department has a unique building and a unique budget. Note two different budgets for the same department and no two different buildings for the same department and we also assume this is not essential, but if we assume that an instructor has only one department. So, these things that these pieces of information are what helps us in the design. They are what helps show that there is a problem with this design. If any combination is possible, if you say that Einstein is in physics department, he is in building Watson and the budget for Einstein's physics department is 70000. But if you go down further, gold is also in the physics department, but the budget for gold could be something else, which does not really make sense in this particular example. But there are other examples where there is no such dependency. Here, we said that department determines the building and the budget, but if it does not, if the department plus the instructor together determine the building, then this design we would get is different. In fact, that is what happens in our CS department here. We were in one building. There was a school of information technology here in another building and then we merged and the two departments became one, but there was not enough space in this building. So, we continued to occupy another building. For other reasons, we also have offices in a third building and soon we are going to get a new building, which will temporarily cause the number of offices to go up to four buildings before it will come down to two when we vacate two other buildings. So, the functional dependency department to building, this is called a functional dependency, which says that department has a unique building does not actually hold for our CS department here at IIT Bombay. But supposing it does hold, you can see that there is repetition of information. Once you say department name is physics and the building is Watson. If physics appears again, the building had better be Watson. Otherwise, there is a mistake in your database. Similarly, if you say the budget is dependent only on the department, then the moment you say that physics has a budget 70,000, if there is another row here with physics and a different budget, there is a problem. So, the functional dependency says that the department here uniquely determines budget. So, the same department appears again anywhere in this relation, then the budget value had better be the same. But the problem with this is, with this particular design is A, it repeats the information. B, it allows for errors. It allows for physics to be associated with 70,000 here and down here, physics may have a budget of 80,000 and it will allow you to update the database and land up in this state. And then you do not know what is the right one. You know something was updated. You know whether this copy of physics was updated and this is the latest or this one was updated and it is the latest. You do not know. So, you have a problem. So, that was a case where a combined schema had a problem. Now, here are two other schemas which we actually got during the ER modeling phase. So, in the ER modeling phase, we had a section week entity and in order to create a relation for the week entity, we had to take the primary key of the identifying entity which is the course. So, we got course ID. Then we had section ID semester year. So, this was the relation we got. We also created a relation for the relationship sec class between section and classroom. Now, what are the attributes of sec class? It had the, this was a mistake. I am sorry, I should have updated this. I saw this mistake during the coordinator's workshop and did not fix it. I will fix it and put the updated slides later. The mistake here is that section class should have the primary key of section which is all of these attributes plus the primary key of the, sorry, the relationship is between a section and a classroom. So, the classroom has a primary key which is building and room number. So, sec class actually has all of these attributes. So, if we combine those two relations into one, this combined relation has all the following attributes, all the attributes of section and all the attributes of section class. In fact, in this particular case, section did not have any extra attributes. Section class would have been a superset and it would be this basically. Now, when we do this combination, there is really no repetition of information. Why? What do we mean by saying there is no repetition of information? So, is any attribute determined solely by something else? Is the room number determined by building? No, it is not. The building has many room numbers, but you know, it is not, the room number is not determined by building. Similarly, building is not determined by rule number and so forth. What about this combination? Course ID, section ID, semester year and then building and room number. So, can you have two tuples with the same value of course ID, section ID, semester year? If you could, then building and room number should be the same. So, if you could have two tuples, that would be repetition of information. But, if you remember our primary key constrained on section, the primary key was course ID, semester year. So, these four form a primary key. So, no two tuples can have exactly the same value for these four attributes. If they did, then their building and room number had better be the same because again, we are assuming in our real world model, a particular course section will meet an only one classroom. It will not meet in two different classrooms. So, under that assumption, these four attributes together form a primary key. If not, there would be, you know, two classrooms associated with the section and then, the primary key would have to include all the attributes here. So, that is for combining schema. So, if you combine schema, sometimes you have errors cropping up. But, there is another option which is to decompose schema. Supposing we started with this bad schema in step. Here we are, this is, we are calling this in step, which has instructor attributes plus department attributes. Now, what we said is that this schema is bad. But, supposing we decompose this schema into two things. On what basis do we do this decomposition? Now, we know that if we actually broke it up into instructor and department, the schema is good. But, that is because we started with that. If we just gave you a relation like this, on what basis do you break it up into multiple relations? And the basis is what is called a functional dependency, which basically says, take the intuition here. If there were a schema department name building budget, then the department name would be a candidate key. In other words, department name uniquely determines the budget and the building. That is a property which constraint we are assuming on the real world. If the real world represents this constraint, then we can make this statement. And based on that statement, we formalize this statement incidentally by saying there is a functional dependency from department name to building budget. So, the right arrow here says that the left hand side determines the right hand side uniquely. If you had two different tuples with the same value of department name, they better have the same value of building and budget, because the department has a unique building and budget. So, that is the idea. Now, given this dependency in this combined relation in step which we saw, department name is unfortunately not a candidate key. What this means is, we are allowed to have two different tuples with the same value of department name. If you did that, immediately building and budget are going to get repeated. So, this functional dependency shows us what is wrong with that original schema. The fact that there is such a dependency and at the same time department name is not a candidate key. In other words, it is not unique. It can appear more than once in that relation, then there is a problem. And how do you solve the problem? You decompose. So, the decomposition is what we are already familiar with, instructor and department. So, this decomposition was good. On the other hand, there can be bad decompositions which lead to problems. So, let us take this relation, slightly extended version of employee. In our version of employee, we had ID, name and salary.  It does not really matter too much, but let us add this.  One relation is employee one ID name. The other relation is name, street, city, salary. So, when we decompose, we are breaking things up, but we do not want to lose information when we decompose. And in fact, this particular example, if we assume that names can repeat. So, what is our dependency here? An employee ID is unique. And the employee ID determines the name, the street, city and salary. Given a particular employee ID, these are, there cannot be two different values for any of these attributes. They are uniquely determined. However, we are not saying anything about name. There could be duplicate names. That is realistic actually. So, this particular decomposition is actually what is called a lossy decomposition. It loses information. So, let us show this by means of an example here. So, here is an original relation. There are many rows. Let us focus on these two rows in the middle here. So, there are two employees with different IDs. One is 5776. The other one is 98776. Now, both of them are named Kim. Why Kim? Well, in Korea, apparently, half the people have a surname Kim. So, it is a very common name. People are likely to have the same name. Now, here we have just stored one name. But even if you stored first name, last name, in Korea, there would surely be a lot of clashes given that there are many, many, many people named Kim. In India, we have equivalent with Gupta and Shah and Desai and many other common surnames. There are many people with exactly the same name. I think in the down south, Srinivasan is a very common name. We had Suresh was a very common name. So, in my class, we had in engineering, we had, I think, maybe five Ramkumars, two of whom had the same initials. So, clearly clashes are possible. Now, those two Kims live in different places. So, their street and city may be different. Their salaries may be different. Now, this was the decomposition we decided on. ID name in one relation and name street, city, saloon, another. So, how do we create this decomposition? We take these two tuples and project them on ID name to get this. So, the contents of ID name include these two rows, 5, 7, 7, 6 Kim and the 9, 8, 7, 7, 6 Kim also over there. And similarly, if we project on name street, city, saloon, we get these two tuples. So far, so good. Now, if you want to combine this information and get the information back, we want to get the ID name street, city, saloon of these employees after the decomposition, how do we combine relations? We do a join. In particular here, because we did a decomposition, a natural join is the natural way to combine the information back. So, we do a natural join. What is a natural join? It makes sure that common attribute names have the same value. In this case, name is the common attribute. So, we match tuples, which have the same value for name. Now, unfortunately, both these tuples have the same name. Both these tuples also have the same name. So, what happens is this first tuple matches the first tuple here, which was intended. But the first tuple also matches the second tuple here and we get this extra tuple down here. And similarly, the second tuple matches both those. And what have we got back? We have got four tuples where we had two. Now, you might think, hey, there are more tuples. So, we actually have more information. But in fact, that is wrong. Just because there is, there are more tuples, does not mean the information is more. What has happened is we have lost the information that this particular ID lives in this street, this city and this salve. If you look here, this particular ID is associated with two different addresses and two different salve. So, we have lost the information that this ID is associated with actually just this one street city salve. So, although we have gained extra tuples, we have lost information. We did not lose tuples, we lost information. That is why this decomposition is called a lossy decomposition. In general, whenever you decompose and join back, you can only gain tuples. Any tuple that was there in the first place would be regenerated. The only effect is you get extra tuples. So, lossy in this context means you get extra tuples, but you lose information. So, that has been. Now, here is another small example where we are simplifying things. And in fact, we are going to use small examples like this in this chapter. Why? Because if you keep long attribute names, the examples become long, the text overflows into multiple lines. So, we are going to rename attributes as A, B, C, D, E, F, G and so forth. We are going to abstract it away. So, if you have a real world thing, we can just assign letters to the attribute names and then work with those letters. We are going to work on relation schema. So, the schema in the schema, the relation name will be, well actually the schema is the set of list of attributes in the relation. We will use capital letters. We sometimes use the notation R equal to A, B, C. Sometimes we just say R of A, B, C, meaning the schema R has the attribute A, B, C. Now, that is the schema. This is a particular instance of that relation. So, here we have alpha 1 A, beta 2 B as the two tuples. And if you project it on to A, B and B, C, what do we get? Alpha 1, beta 2 is the first one and on B, C you get 1 A and 2 B, so far so good. Now, if you join these back, what happens? You get again it is a natural join, the common attribute being B. So, you get alpha 1 matched with 1 A, because B is repeated, it is only shown once. You get alpha 1 A. Similarly, beta 2 B is the next tuple. So, what we got is exactly the same relation back. So, in this case the decomposition is lossless. But, can we say that this is a particular instance of R? So, supposing we take a particular state of the employee database as it is today and we do this decomposition and join it back and say everything is fine. Is that good enough? Well, let us come back here. In this case, the problem was shown because of two employees with the same name. But, supposing in our database, there were no two employees with the same name at the current point in time. This may change tomorrow. You may get another new employee with exactly the same name. For a long time, I was the only Sudarshan in the faculty at IIT Bombay. I mean, there were other Sudarshan before me, but when I joined there was no other Sudarshan. And a few years back, another Sudarshan joined, which of course cost a fair amount of havoc with letters meant for him or calls meant for me going to him or to me. There was a lot of mix up for a while until people realize there are two Sudarshan. Anyway, coming back, at this point it is fine, but the moment you get two rows with the same B, the decomposition is not lossless. So, what we want is some constraints which will be guaranteed to hold. It is not just that they hold on the state of the database now, but that they will hold on any valid state of the database. So, that is something which we are going to go into a lot of detail in just a few minutes from now. Now, before we get into that, there are some other things which we need to address. So, using this notion of functional dependency and certain other dependencies, database theoreticians starting from Codd himself. As I told you, Codd was the father of the relational data model in some sense. There were others before him who had proposed it, similar things, but he put it together, came up with a whole theory including the theory of functional dependencies and so forth. A lot of the stuff which we are going to study today was pioneered by Codd, thus added on to it later. So, now they defined a series of normal forms. They say that a relation is in a normal form if it satisfies certain property. And the normal forms were defined starting with some very simple properties which you absolutely want relations to satisfy. And then some more stringent properties which occasionally you might allow things to violate for efficiency reasons for some other reasons. So, the first normal form, this is the basic principle which Codd said that he wanted domains to be atomic. What does it mean for a domain to be? First of all, what is a domain? A domain is a set of values that a particular attribute can take. And we said that a domain is atomic if all the elements in that domain or all the specific values which are which an attribute is allowed to take are indivisible. So, what are examples of things which are atomic? Normally, you would think that a salary value is atomic. You cannot break a salary number into parts. Now, you can break a salary overall into parts, basic D A and so on. But if you keep a single number for the salary, you cannot break that up. Similarly, if you have maybe a city where the person resides, you cannot break the city up. Well of course, cities have been broken up, states have been broken up, countries are being broken up. But that aside in the database, you cannot break the name of the city into parts. On the other hand, what are things which are not atomic? Name in particular a set of names. So, supposing we had employee record and we keep a set of dependence as one of the attributes. So, it is a multi-valued attribute. So, if you sort a set of names of children, then it is indeed divisible. Each name in that set is a separate thing. It refers to a different child. So, it is not atomic. What about composite attributes? We had name which consists of first name, last name, middle initial and so forth. Is that atomic? Actually, it is not. It can be broken up into parts. So, in the relational model, when you create actual relations, you break it up. But this is actually not such a big deal. Composite attributes, you can just pretend, you know, it is sometimes useful to refer to them as one and sometimes useful to refer to them as part. So, we can think of having individual attributes, first name, last name, middle initial and then name is something which is a convenient way to refer to all three attributes put together. So, composite attributes are not really a problem in this context. But there is another more insidious problem which people often do not realize. If I have a course name like CS 101, if I can break it into CS and 101, then it is not really atomic. Why? What do I mean by if I can break it? Obviously, I can take the first two characters in the last three characters and I am breaking it. But the question is, is this break up meaningful? And in this case, if you say that CS refers to computer science department, then pulling out the first two characters of the course gives you back meaningful information. So, in some sense, this course ID is not atomic. It can be broken up. However, we do use course IDs like this and the point to note is even if it can be broken up as long as you are careful about not breaking it up and using the broken up parts in your application and in your queries and so forth, then it is ok. So, we treat it as inducible although human might break it up. In the database, we do not pull out two characters and do stuff with it. Now, that was not always the case in IIT Bombay. A while ago in our early databases, people said why store department separately with course ID? We will just pull out the first two characters of course ID and then interpret it as a department code. This is not really a good idea because at some point, we wanted to add course IDs which were three characters and then the code which broke it into two characters stopped working and there is a bit of a mess there. So, this is one example of why you should not be breaking it up. Now, if you had stored a relation separately which links a course with the department, just look up that relation. What is the big deal? Just one look up on that relation will give you the department of a particular course code. So, do not do string manipulation on it. Now, what is the problems with non-atomic values? Why was called against non-atomic values? There were several reasons. First of all, it complicates storage, it complicates querying, it complicates query optimization and so on. So, one of the reasons a relational model succeeded was because it was very, very simple. Because it was very, very simple, creating systems to store relational data was relatively easy. Because the thing was very simple, it was relatively easy to come up with query languages for it and because the query languages were also relatively simple, it was possible to build optimizers which could take a query which was written in the query language and then figure out how to do it efficiently. If this had not been the case, then query optimizers would have been terrible and people would not have used SQL. They would have gone back to writing code directly on some underlying API and not used SQL and then lost all the benefits of SQL. Now, this is actually, it is a really, really big deal. It is not obvious, but this is a big deal. You keep things simple, then you can get other benefits and this has been proven multiple times. So, the first time around that this was proven was when people said, but why should we stop with simple relations? Why not add features to relations? Let us add such value attributes. Let us add objects. Let us add inheritance and so forth. So, they added a lot of stuff. They created object-oriented databases. They added object-oriented features to SQL databases, but it did not take off. One of the major reasons it did not take off was once you added all this complexity, it was actually very easy to write queries which the optimizer really could not execute very efficiently. So, you write a query which seems like a logical query, something which should run reasonably efficiently, but because the language is complex, the optimizers could not do everything that they ought to do, could take a lot of effort to make them powerful enough, which was not done initially at least.  I do not want to use it. It is very, very slow. Let me stick with relational databases. So, many of these things, people put a huge amount of effort building them and then they really flopped at the marketplace because practitioners who tried using them found the performance was terrible. This history repeated itself with XML databases when they were first launched. Again, the performance was terrible. People tried it out and said, no, no, we are not going to use this. We will stick with relational databases. Later on, companies put a lot of effort into XML storage and databases and actually made them work quite well. However, by then people had burned their fingers and did not want to go back to it. So, it was too little, too late in some sense. So, the simplicity of the relational model played a huge role in its success and we should not forget this. So, that was with respect to query languages, with respect to storage also. If you have non-atomic values, how do you store it physically? Well, there are ways of doing it. In fact, if you think about it, we saw multivalued attributes in the ER model and we saw exactly how to convert it to relational model. We simply broke it up into a separate relation and stored multiple tuples. And this is eminently doable even if you start with the multivalued attribute in the relational model. You can break it up into a separate relation and store it. All that chord said was, do this directly and expose the underlying schema. So, we are going to assume that all relations are in first normal form. Now, here is another example. I talked of course, IDs, but a similar example repeated itself in IIT Bombay, where students are given roll number, which they do not look exactly like this. But essentially, somewhere in the roll number, there is initially a year followed by a department code.   And guess what happens? People do change branch. People do change MTech students who joined a self-finance. Sometimes, there are teaching us, then seats unfilled sometimes and we give it to them and they become teaching us. And guess what academic office did? They went and changed the roll numbers. Fantastic. So, at the beginning of the course, I have a list of roll numbers, I conduct exams, I give marks based on those roll numbers. And at the end of the semester, the roll number has changed. The academic office would give me a list of roll numbers for which I should give grades. And there are roll numbers here who never took the first few exams. Now, what on earth do I do? How do I know the mapping? It is madness. I mean, we complain loudly to academic office and eventually, we force them to stop changing roll numbers at least. But they did not give up the practice of encoding all these same things in the roll number, which is actually very bad in spite of me as a database person going and telling them, look, this is bad database design. They refused to listen. Even today, we have exactly the same scheme there. The only thing that they do not do is change the roll numbers in the middle. And then, we have all kinds of problems with people looking at a roll number and saying, oh, this person is in EE department, but actually this person is in computer science department because they had a branch change. Now, in the database, it is not a big deal. Just go do a look up. Why do you have to do all this kind of stuff? Anyway, so it is a very bad idea to encode these kinds of things, particularly in primary key because changing the primary key involves all kinds of updates. If you have to change a primary key, there are many things referring it. In the example I told you, the marks which I have for various exam are not in the database, but they are actually foreign keys referencing the roll number. And if you change the roll number underneath me, I am in trouble. The update to the roll number should be cascaded to all things that refer to that roll number. So, it becomes a potentially expensive update. So, this also brings up a general principle for choosing primary keys. You really do not want to choose primary keys using attributes which might change. The attributes might be sufficient to uniquely identify an individual, but if it is going to change, do not choose it as a primary key. For example, name and address would probably uniquely identify an individual. It is unlikely there are going to be two people of the same name and the same address. So, you could choose that as a primary key, but names change, addresses change and then it has cascading effects on all things that refer to it. So, as part of database design, choose the primary key wisely using things that are not going to change. So, coming back up to a theoretical side, we want a theory which does the following. It decides whether a particular relation is in good form. We saw the first normal form. That is a very basic thing. There are many more normal forms coming up. And if it is not a normal form, decompose it into a set of relations such that each relation is in a good form. So, I want to break a given relation into pieces. Again, this reflects the history where people said, let us start with a single relation containing all the attributes of interest from everything that we want to model and then break it up. So, we start with that mega relation and then break it up if you follow that theory as is. However, we generally do not follow that theory in that direction. We end up doing air modeling and land up with a set of small relations, but we can still apply the theory on the smaller relations which we get. So, our theory is based on functional dependency.  Functional dependency is one kind of constraint on the set of legal relations. What do we mean? We mean that any legal relation, legal meaning it is allowed in the real world. Any legal relation must satisfy some properties and these are the constraints. And functional dependency is one kind of such constraint. There are many, many other constraints. We have already seen primary and foreign key. That turns out primary keys are really a special case of functional dependencies as we will see. And the functional dependency requires that the values for a certain set of attributes determines the value for another set of attributes uniquely and it generalizes the notion of a key. So, let us take an example. If I have a set of attributes capital R, we will use capital letters to denote sets of attributes and we will use Greek letters, alpha, beta and so on to denote subsets of these. Whenever I say subset, I generally mean subset of or equal to. So, as a special case alpha could be equal to R, beta could be equal to R. Now, we say that given a relation schema R, we will say the functional dependency alpha determines beta holds, holds is the key term here. If and only if for any legal relations R on R. So, what is this notation? R denotes a relation and instance on this schema capital R, any legal relation, anything which is legal in the real world. If any legal relation, whenever any two tuples in that relation have agree on alpha meaning they have the same value for alpha, then they also agree on beta that is they have the same value of beta. In other words, if you have two tuples T 1 and T 2, if T 1 alpha equal to T 2 alpha that is the projection of T 1 and alpha is exactly the same as the projection of T 2 and alpha, then the projection of T 1 on beta must be equal to the projection of T 2 and beta. Now, take this very small instance R A B with A being 1, 1, 3 and B correspondingly 4, 5, 7. Does A functionally determine B hold? It does not because we have these two tuples which have the same value for A, but they do not have the same value for B. Therefore, A determines B does not hold. Now, on this instance B determines A does hold, there are no two things to the same B. So, trivially you know this condition is never invoked and B determines A holds on this particular instance. That does not mean that it will hold on all legal relations, all we are saying is on this particular example instance the functional dependency holds. Now, in general what we want is functional dependencies on schemas which means that it holds on all legal relations. So, that is what we are going to focus on. Although we can say what functional dependencies hold on a particular instance that is usually irrelevant to us. So, we are from now on when we say a functional dependency holds, we are only going to think about or refer to functional dependencies that hold on the schema. We do not care about anything that holds on a particular instance, but may not hold on another instance. So, now we can define super keys easily using functional dependencies. K is a super key for relation schema R if and only if K function determines R. So, if two tuples have the same value of K, they must agree on all attributes of the schema. In other words K uniquely determines a tuple in R. Now, candidate keys can also be defined formally using functional dependency. So, we can say K is a candidate key for R if and only if first of all K functionally determines R that determines all the attributes in the schema. And moreover for no alpha subset of K, thus alpha functionally determine R. If it did then alpha would also be a super key and then K would not be minimal because there is subset of K which is also a super key. And therefore, if K is not minimal it cannot be a candidate key. In other words what this is saying is K is a candidate key. If it is a super key and it is minimal no subset of it is a super key. And what about primary keys? Well it is basically one of the candidate keys is chosen. How do we choose it? The theory does not define it, but as I told you we should choose things which will not change. Now, we saw in depth earlier and we saw some intuitive things about it which said department determines building and budget. We can formalize it using functional dependencies. So, for example, here we would have the dependency building sorry department name determines building that in the real world we are assuming a department has a unique building. We would also have department name determines budget it is not shown here, but it would hold. You would also say that ID determines building. Why? Is that obvious? Well we know that department determines building. We also have assumed in our model that an employee is in exactly one department. Therefore, we can say ID determines department. If ID determines department and department determines the building then you know given a particular individual they cannot be in two buildings. So, we can infer that ID determines building. So, what I have just shown you is an intuitive notion that if I am given two functional dependencies I can actually infer another functional dependency and we will see this formally coming up. However, we would not expect the following dependency to hold department name determines salve. Why? Because there may be two different people in the same department with two different salve. So, just because two records have the same department name in this in depth relation we cannot say they have the same salve. So, let me finish this slide and then we will have some time for questions after this. So, what is the use of functional dependencies? As I said we can test relations to see if they are legal and we will say that a relation are satisfies F if R is legal under a set of functional dependencies F. So, this by relation here we could be referring to relation instances, but the part which we are really going to use is that F holds on R if all legal relations on the schema satisfy this set of functional dependencies. So, this is formally there is a distinction between instant satisfying and holds, but since we really do not care about instances going forward sometimes we are loose with terminology. We will say satisfies or holds regardless of what we say we mean that holds really that is it is true of all legal relations plus bullet here we have already seen. So, I think this is a good point to take some questions before we go into the theory of functional dependencies. I am seeing a few questions on chat. Let me take a few of these questions and then I will come to live questions. One of the questions says first normal form says values must be atomic then how come normalization supports multi-valued attributes. Normalization does not support multi-valued attributes. The whole idea is you start with a relation in the first normal form and then apply this theory because a functional dependencies and so on do not really apply to multi-valued attributes. There is a notion of multi-valued dependency which we will look at later, but the formal theory of multi-valued dependencies is also done in terms of relations which are already in the first normal form. So, the basic assumption is that we have already got stuff to first normal form. How do we do it? We know how to turn multi-valued attributes into some flatten it, but if you want a universal relation model you actually just flatten it into one relation with all the attributes. So, from there we start the normalization process. The next question says the domains of all attributes atomic. Now, while designing tables in some real world application the data type is where care and thus allows us to store more than one telephone number in a single set thus atomicity is lost. Can we say that such a table is in first normal form? That is a good question. Once you have a wire care you can put anything in there, you can put comma separated lists and create sets and so on. The relational model is not able to do anything about that. However, if you see the operations which are there in the relational model there is no basic operation to split an attribute into parts. Now, an actual language like SQL for practical reasons allows you to do such splits, but the model itself does not allow you to do anything with those kinds of splitting. So, if you start interpreting the values your relation is not in first normal form. If you do not interpret it and break it up then it is in first normal form. So, in some sense being in first normal form is a property of how you use it not just of the raw data which is stored there. The last question I am taking off chat says can we combine two schemas if they have different sizes of tuples. What I think you mean is that the number of attributes in each tuple is different. Yeah, it does not really matter. We can combine the schemas. How does it matter? Now, let us take a few live questions. Sushila Danchan, Maharashtra, please go ahead. What is the key value and what is the key column and non key column in functional difference? So, the question is what is the key column and what is the non key column. So, a key is often used by many authors to mean a candidate key. A super key is something which is a super which uniquely determines and many books and papers when they say key they mean a candidate key. So, we just define what is a candidate key. It is a super key which is minimal. No sub part of it is a super key. So, that is what a key is and we can define a key as we saw in terms of the functional dependency. We have a CCOM engineering college West Bengal. CCOM, do you have any question? What is spurious tuple? So, I am not sure. We do not use that term in a book and some of these terms have different meanings depending on who uses it. Spurious generally means something which is not real. It comes in as an artifact of something else. So, for example, if you have decomposed incorrectly, if you join and then get some extra tuples in the result, that could be considered as spurious tuple in the join after a bad decomposition. That should not have been there, but because you made a mistake in decomposition, you get extra tuples which are spurious tuples. That is probably the context in which you saw it. Let us move on to theme college Maharashtra Boisar, theme college, go ahead please if you have a question. Sir, actually in first normal bomb, we do not require the multiple values. So, here we are giving the primary key. What do we do exactly the super key? So, I think the confusion here when we said multiple values, what we mean is multiple values within a single attribute. I think the confusion here is when I say multiple values, you may be thinking that you cannot have a super key having two attributes. No, that is not the point at all. The point of multiple values is in a single attribute, if you store a set, then there is a problem. It is not in first normal form, but there is no problem having a key which has three different attributes. So, of course, each attribute has its own value. So, there are three values, one per attribute. That is fine. That is not an issue at all. Does that answer your question? Sir, one more question. What do you buy determinant in functional dependency? So, there are again different ways in which this term determining is used. So, for example, you may look at the real world and determine, figure out that a particular functional dependency holds. You can also use the term in the sense that if you have a functional dependency, a determines b. You can say that value of a determines the value of b, meaning that if I am given a value of a, there cannot be two different values for b, they can only be one value of b. So, it is used in that sense also. So, maybe what I am seeing is, people are asking a lot of questions about things they want to know and I am going to cover those. So, maybe I should just continue with my next few slides and those will answer many of these questions. So, what we are going to do now is look at theory of functional dependencies to some extent. We are going to go deeper into it later. So, first of all, we would say that a functional dependency is trivial if it would be satisfied by all instance, all possible instances of a relation. Let us say that there is no constraint at all on the relation. Any value at all is possible. In spite of that, a class of functional dependencies called trivial functional dependencies will always be satisfied because the right hand side is a subset of the left hand side. So, this is an example. ID comma name determines name, functionally determines name. This is kind of obvious. If you take two tuples in any possible instance, if they have the same value for ID and name, obviously they must have the same value for ID, I mean it is trivial. Similarly, name determines name because if two tuples have the same value for name, of course, they will have the same value for name, it is obvious. So, in general we will say that the functional dependency alpha determines beta is trivial if beta is subset of or equal to alpha. So, why do we even care about this? Well, when we do various things, we will exclude trivial dependencies because in some sense they do not matter. They are obviously going to be true always. So, we will exclude them.