Hello and welcome my name is Shannon Kemp and I'm the Chief Digital Manager of Data Diversity. We hope everyone is staying safe and well out there and we would like to thank you for joining the current installment of the monthly Data Diversity Webinar Series, Real World Data Governance with Bob Siner. Today Bob will be discussing Data Governance and Policy Management. Just a couple of points to get us started due to the large number of people that attend these sessions you will be muted during the webinar. If you'd like to chat with us or with each other we certainly encourage you to do so. You can click on the chat icon in the bottom middle of your screen for that feature. And for questions we will be collecting them via the Q&A in the bottom right hand corner of your screen or if you'd like to tweet we encourage you to share our answer questions via Twitter using hashtag RWDG. And if you'd like to engage more with Bob and continue the conversations after the webinar you can go to the dataversitycommunity at community.dativersity.net. As always we will send a follow-up email within two business days containing links to the slides, the recording of the session and additional information requested throughout the webinar. Now let me introduce to you our speaker for the series, Bob Siner. Bob is the president and principal of KIK Consulting and Educational Services and the publisher of the data administration newsletter T-Dan.com. Bob has been a recipient of the Dama Professional Award for significant and demonstrable contributions to the data management industry. Bob specializes in non-invasive data governance, data stewardship and metadata management solutions. And with that I will give the floor to Bob to get today's webinar started. Hello and welcome. Hi Shannon, hi everybody, I want to echo what Shannon said and I hope everybody is staying safe and staying well, especially during these difficult times. But it's great to have you here and on this monthly, this month's installment of the webinar series, as Shannon mentioned the name of this webinar is Data Governance and Policy Management. Now if you've been a listener or been an attendee of these webinars over the years you know that I've done several webinars on data governance policy and so we're not going to specifically, well maybe a little bit, we'll be talking about specifically what it takes to build out a data governance policy but we're going to talk mostly in this webinar about the data governance, about data governance as a discipline and its relationship to policy management. So I'll refer to the other webinar, you may see a slide or two that talks to some of the things that I might have mentioned in the previous webinar about how to build out a data governance policy. Before I get started I'd like to share with you a bunch of things that I'm involved with and I tend to be involved with a bunch, well certainly there is this monthly webinar series and in July, and so you know it's on the third Thursday of every month at 2 p.m. In July I will be talking about building a data governance roadmap and there's lots of ways that you can register for the webinar. I talk a lot about non-invasive data governance and want to refer you to the book on non-invasive data governance that can be found at your favorite booksellers everywhere I think. Also speaking, I'll be speaking at a couple of dataversity events that are coming up and our prize data world will be in Chicago this year in October and the data governance and information quality conference, DGIQ, will be in Washington D.C. in December. I do have a brand new piece of information to share with you and I think Shannon's excited about this and I'm certainly excited about this. I talk about the online classes that I provide the non-invasive data governance one and the metadata governance one as well but we have one that we're going to be introducing in August called business glossaries, data dictionaries and data catalogs so please keep an eye out for that. That's a great course about you know building requirements for these tools, what goes into these tools, governing the metadata in these tools so that'll be a great a great learning plan that'll be available pretty soon. Shannon mentioned the data administration newsletter tdan.com that's available always at tdan.com and it publishes twice monthly and it's always the day before the webinar since I'm on the third Thursday, the day before I always publish new content so the new issue of tdan is out there and last but not least is kik consulting and educational services, kik stands for knowledge is king so the focus is on knowledge transfer and I call it the home of non-invasive data governance because that's where I do a lot of my writing and a lot of my work so again good to have you with us today we'll be talking about data governance and policy management the the subjects that I want to talk about in this webinar are going to be the relationship between data governance and policy management you know how to take a look through your organization to recognize what data policies are already in place they may not be called data governance policies but I can almost assure you that if you're a fairly sizable organization that you have data policies already in place in your organization we'll talk about using data policy to bolster the data governance program and then like I said I will spend a little bit of time talking about the makeup of a data governance policy or a policy that would be called a data governance policy and then we'll talk about having a policy in place to manage your policies so that's important and it's very important that we share information about the policies that we have in place because and then if you think about trying to be as non-invasive as you can in the governance you're going to look to whatever already exists within your organization and even though you might have an IT security policy that's called an IT security policy it may in fact be kind of a data governance policy and almost any policy that you have is a governing policy that's the reason they're put in place but we'll talk about having a policy in place to manage your policies so for the first part of this course I want to talk specifically about the relationship between data governance and policy management and I'm going to start out by defining what data governance is what policy management is and how these things are similar how these things are different and how we can kind of tie these things together so they make sense so we we make certain that we're governing our policies that to almost the same extent that we're governing our data we need to make certain that we have people that are accountable for the policies and making sure that they're being enforced just like we need to have people that are responsible for data governance and there's a need to enforce that as well across the organization so I'm going to start out with my definition of data governance and it's worded very strongly and it makes some people cringe a little bit but the idea is that it is important to get people to take notice of what you mean by data governance so I define data governance as the execution and enforcement of authority over the management of data and data related assets so execution and enforcement of authority those are worded pretty strong but the fact is that at the end of the day no matter what approach you take to implementing a data governance program the goal has to be to execute and enforce authority over the management of data the idea is to get people to do the things that they need to do to improve the definition the production and the usage of data across the organization so I like to word it strongly I've had clients that have tempered the definition a little bit but execution and enforcement of authority there are some clients who say you know what we need to keep that as the wording for it because it is does have a little bit of muscle behind it a little bit of grit your teeth sit forward in your chair and take notice what do we mean by executing and enforcing authority so it is worded very strongly it gets people to pay attention some organizations love it some organizations don't like it as much and they have you intending to temper the definition of data governance to more of the formalization of behavior associated with data which is good because that's really what we're trying to do excuse me but you know what we need to make certain that we enforce authority over the management of data and the formalization of behavior when I define the term data stewardship which I'm not I don't have a slide on that in this webinar but data stewardship is the formalization of accountability for the management of data and data related assets so people who define produce and use data if we're going to hold them formally accountable for how they define produce and use data that they are stewards of the data they can't say no I'm not I'm not going to protect data that is sensitive they are a steward and I always tell people that we kind of need to get past that fact and understand that potentially everybody in the organization could be a data steward so those are my definitions that's really my definition of data governance but what is my definition of what policy management is and I thought that that this would have to have been worded somewhere better than what I would think of for the definition I'd use a cheeseburger definition and I'll refer to that in a minute but I went to policymedical.com and they had a really solid definition of what a what policy management is and so policy management is the creation all right is is the process of creating communicating and maintaining policies and procedures within an organization so you know the fact is that if we're going to have policies we're probably going to have more than one policy they're going to come from different parts of the organization they need to be approved by people in the organization so needing to have that process to create and to communicate and maintain you know both policies and procedures within an organization I just thought that that was a pretty much spot on definition of what policy management is and you know again left to my own own devices here I would say well it's the management of policies right and I always tell people not to use cheeseburger definitions a definition of a cheeseburger is a burger with cheese but and we don't want to say that the policy management is the management of policies you don't want to use the same words of what you're defining in the definition so that basically a policy is a code of conduct and it is something that we are expecting people in the organization to abide by and then to follow so oftentimes when you create policies they need to be filtered down to govern the enterprise and the divisions and the regions and the business units and the processes within the organization so oftentimes policies are developed but then they need to be communicated to people because people need to be know need to know what their responsibility is in association with that policy and how the policy is going to be enforced across the organization so the policy is a great thing to have but you don't want to have a bookshelf full of policies that people don't know or they don't understand or they don't follow if you do that then you're going to have a bunch of documents that have taken a lot of time and effort and have required senior level approval yet they're not doing for the organization what they really need to do for the organization so I wanted to compare the definition of policy management to the definition to a definition of data management that's kind of based on the definition of policy management and then kind of throw in the definition of data governance after that as well but I said that the policy management is the is the process of creating communicating maintaining the policies and procedures within the organization well you think about it data management and I know a lot of you are data management practitioners your data management is the process of creating communicating maintaining data and metadata procedures within the organization so hoping to improve how the organization defines their data produces their data and uses their data so those two definitions I actually just stole from the policy management one inserted data management and then inserted data and metadata procedures instead of what they said policies and procedures so really policy management is very similar to data management they're different in the fact that they manage different things but they could be defined the same way they are that process of creating communicating and so on within the organization now if you compare it to data governance and you could almost take the policy management definition and say that data governance is the process of creating communicating maintaining and executing and enforcing rules and behaviors associated with data and metadata procedures within an organization you know that would be a valid definition for an organization to use that that is really what data governance is all about it kind of takes the best of the policy management definition and the data management definition to define data governance so again it's not a definition that I use regularly but I think it sits in the narrative of what we are talking about today and so you can see there's definitely a relationship between data governance and policy management but we also want to ask the question well what is policy management practice you know what will it do and we and certainly it moderates risk for your organization in a couple different ways so the effective policy management makes policies more accessible to your staff for guiding their activities and the safety and safety decisions that are made within the organization I'm not necessarily talking about people safety but safety for the organization also an effective policy management practice protects an organization from litigation by staying up to date on standards and creating an audit trail in case of legal action so having a policy in place many organizations like I said before have information security or IT security policies they have data protection data privacy policies and what are they doing they're setting their organization up to have that audit trail in case there's legal action and that's where data governance programs get started is they're kind of being spurred on by the fact that we have a policy that says that we need to do things that perhaps we don't do as well as we could you know that we need to do that moving forward so let's look at the similarities between governance and policy management they're both codes of conduct they both require that their folks are down throughout the enterprise down from the enterprise to the divisions or departments or lines of business whatever you call them within your organization to the different regions of the country and of the world you know there's different policies that need to be in place for different regions of the world because the rules are different in different places we know that GDPR started in the in the European Union I believe and some of the local some of the California Protection Act is very localized but you know it changes region to region and we need to make certain that our policy our policy and the data governance is able to adapt depending on the lines of business depending on the regions the business units and the processes and both of these things require leadership and approval I mentioned often that the very first best practice that organizations use are that the data governance the very first best practice is that senior leadership supports sponsors and understands the activities of data governance because at some point you're going to be at risk if they don't support sponsor and understand what you're doing well the same thing can be whole help true for policies you know they both require leadership they both undoubtedly require approval and knowledge as to why these policies are being put into place and they're not going to be approved unless the people at the top parts of the organization who sign off on these policies really understand what they're all about so they both certainly feel invasive unless you sell it otherwise to the organization and you can use the term non-invasive to say okay well we already have policies and we already have people that have responsibility for data we need to formalize that accountability as we move forward so just like comparing apples to apples you can compare apples to oranges and we can talk about the differences of data governance and policy management so I stated that there's a lot bunch of similarities between the two and even in the definitions of the two but there's also differences as well so data governance really focuses on data and information and metadata you've heard data governance you've probably heard information governance I've spoken a lot about metadata governance and by saying that the data the information and the metadata will not govern itself we need to make certain that we execute and enforce authority over the management of those important assets to the organization so that's the data governance side and on the policy management side it focuses on the management of the policies themselves and the information and the information about the policy and even letting people know that these policies exist and that they're expected to follow them so we know for sure that in the relationship between the two they both demand execution and enforcement again that's the reason I like to word it strongly because it makes people pay attention to what we're doing again make sure that you define it the way it needs to for your organization but I like to word it strongly they're both necessary in order for an organization to be disciplined organization so I say it's necessary actually you could say it's required in order for your organization to become a more disciplined organization and I say that there can be policy about data but the fact is that there probably already is policy in place in your organization about data it may not be called data governance policy but you may have data classification and protection and handling policy you know how do we handle data based on how that data is classified how can we print it how can we share it how can we do we need to lock our computer if we're showing that's something on the screen or even printing and destroying and retaining information those are all things that have to do with the data classification and the protection and handling policy there's data security policy there's policy around data sharing and data access many organizations that work with have operational policy associated with data or even policy that states who the owner of that data is if you know me you know I kind of shy away from the term owner because it implies what I say is exactly the wrong thing it implies that the person maybe owns the data rather than the organization the term steward and stewardship and the definition of those are that that's when somebody takes care of something for somebody else it should be more data stewardship policy but many organizations do have data ownership policy and data and metadata change management policy I've worked with organizations that put a lot of resources into the building of their glossary and their dictionary and their data catalog and if somebody's going to want to make change to something there's a policy for how they go about making those types of changes so I would venture to guess that in most of your organizations they already have policy they may already have a lot of policy I've seen binders of policies within organizations so let's talk a little bit now about how to recognize the policies that are already in place and the first thing that I want to address on this is that you know first of all some organizations are policy driven some organizations are not policy driven so let's talk a little bit about what it means to be policy driven and not what makes a policy a good policy where to look in your organization to see what policies you have we'll talk about policies that govern that I'll kind of give it away a little bit here that you know pretty much any policy that you put in place governs that is the purpose for having the policy in the first place and then applying governance to the policies themselves so we talk about defining and producing and using data we certainly need to talk about defining and producing and using policy because it's one thing to define it it's one thing to produce it and get it approved and it's another thing to make certain that it's not shelfware and being used by people across the organization so let's look at the policy driven yes or no and I have a slide for yes and I have a slide for no so don't take by this slide that I say that policy is required for every organization some organizations are policy driven organizations and it's it's necessary to have policy for everything and in fact that might be some of the organizations that actually have data governance policies is because they're so policy driven they need to have a policy that defines the role of the steering committee and the the data governance council or or even the group that has the responsibility for implementing the governance program and the one thing about being a policy driven organization that's really important is that it demonstrates senior leadership support your senior leadership are not going to sign off on policy just willy-nilly they're going to want to make certain that they understand the policy that it's enforceable around the organization so by having policy and having them sign off on the policy it demonstrates senior leadership support and oftentimes the policies outline who's responsible and how we're going to enforce the rules that have been put in place by the policy how we're going to hold people accountable what the consequences are going to be for people that don't follow policy but you might not be one of those organizations that are policy driven so do you require policy well the answer is no and so there's organizations that say we're not policy driven you know we work better with other terminology guidelines and principles and and visions and i'll talk about some of those different classifications of of policy type things here in a couple minutes but certainly in some organizations senior leadership demonstrate their support in different ways they don't necessarily need to sign a policy so again your organization may be policy driven it may not be policy driven it's never a bad thing to have a data governance policy but the question i get a lot is is it necessary to have a data governance policy and you know i give the best consulting answer that i can and that is it depends it depends on your organization but it's certainly uh you know you might find that guidelines work better you know that you're going to enforce your rules that way it's i'm not here to answer the question whether or not your policy driven or not that you can answer that question and determine whether or not a data governance policy per se is necessary or whether all the other policies that i mentioned earlier are necessary or are in place in your organization because you can point to them and say look we do have some governance taking place already and then when it comes to governing behaviors of people um that's why it really becomes more i don't know should i say become the need is to become less invasive and because people have day jobs they're busy and more than a hundred percent of the time and we don't want to pile on them but we want them to help us to manage and to define and produce and use quality data across the organization so let's talk about what makes a policy a good policy and so there's several criteria that i've seen used and that i've used myself and the first one is that it has to be right size for the organization for the size of the business the size of the audience the the size of the topic within your organization you know some organizations utilize policies that they develop internally some bring in policies from the outside if you are a wholly owned subsidiary of a company they may have policies that are in place that govern and guide your organization so we need to right size the policy first we need to understand that within the organization there's a clear hierarchy of different things and a value a value statement strategy statements plans policy standards guidelines and so on all the way down to the operating manuals and the operating procedures within the organization so if you're going to look to develop a policy you want to make certain that it fits in the appropriate place within the hierarchy or if it's even part of the hierarchy before you get focused on it they need to be principles-based and i'm going to share with you the four core principles that i would typically include in a data governance policy and we'll spend a little bit of time on that but they need to be short understood to the point embedded within other things that we do very clear and so i want to provide to you some policy principles that i use often you know when organizations want to start down the path of defining a data governance policy providence needs to be in place that the governing bodies and the authors and the approvers and you might have regulators that say that these things are necessary you want to publicize it you want to publicize the delivery the availability where the people can get access to it you want to train people on it and you want to make certain that they understand that this just has to become part of their job this is policy this is signed off at the top part of the organization if you don't follow policy there will be consequences for that so where do we go in the organization to look to see if there is policy so some organizations have a policy library so you can there's a corporate or an organizational librarian that you may be able to go to and ask them if there are policies or if they can get a list or you can get a list of the policies that are available another good person to go to in your organization to see if there's policies is your records the information management people your rim folks that they've been doing content governance now it's kind of being called information governance a lot of the time but it really is closer to records management and content management but they may know because that's part of their responsibility you know what policies already exist within the organization you can go to senior leaders you can look at orientation and training materials because we might be educating people of the organization as they come on board as to what the policies are and what needs to be followed i know having been through that many times with different organizations that they oftentimes include lists of the policies and how the policies apply to new hires and things like that or you may want to go to your legal department and ask your legal department what do we have by way of policy do we have a data governance policy operating policy you know we want to be able to point to the policy and say you know we it that policies are important for our organization and that perhaps we want to consider building a data governance policy for our organization so these are all places that you can look within your organization to determine what policies exist some organizations even post this information online so let's talk about which of those policies and the policies that i mentioned earlier um govern and i would be i gave it away earlier but the fact is that all policies govern all policies that are in place require the execution and the enforcement and the accountability so all policies that's pretty much the definition of a policy is that it's put in place to govern the organization not necessarily always the data that govern the organization so all policies require kind of consequences for not doing something that's the way the rules take place that's if you break a law that's like breaking a policy there's got to be consequences for those actions i'm not following and we want to make certain that we're applying governance to the policies themselves and that there are people in the organization that have the responsibility for developing policies and taking a look to see what is the structure that we as an organization follow when we're developing a policy you don't want to create something that's totally in left field when it when there is a structure for what policy looks like and you want to make sure everybody's not creating policy you've got the appropriate people in the organization defining what policies need to be put in place and what that policy is going to look like look like somebody needs to govern the production of the policy and the development of it and the bringing it to the appropriate people in the organization to get their approval and to get their sign off and then there's the governance of the policy usage making certain that their rules are being covered or being followed they're being enforced there are consequences for not following the rules and there's the governance over the maintenance and the review of the policy from time to time we need to make certain that this is this policy still accurate do we need to change things does it need to be updated is there a new technology or a new law that is in place that requires us to maintain or review the policy and then socializing the policy that's important as well somebody has to be responsible for it so applying governance to the policies you know you need to govern the definition of data the production usage of data the maintenance and review of data and certainly the socialization of information about the data as well so you want to make sure you're applying governance you know the execution and the enforcement of authority to the policies themselves as well so there are a lot of relationships between data governance and policy management so now let's talk about using data policy to bolster your data governance program so there's data policy governance which we talked about a little bit before data policy stewardship which is again getting people involved in the policy and making certain that it's communicated to the appropriate people there's the policy approval process using the policy to strengthen governance and then we'll take a quick look at what it might look like in your organization would you be able to govern data if you don't have policy or what is the result of policy of data governance if you don't have something that demonstrates that your leadership is behind it so if you look on this screen kind of grayed out a little bit is the definition that i used for data governance the execution and enforcement of authority but let's talk about the execution and the enforcement of authority of the policy and making certain that we have people that are responsible and we have processed for definition of policy production and usage and to make people make certain that people follow them policy for data and data related assets i mean you look into seeing what policies you already have in place and i assure you that you have data operations policies i asked a client of mine if they had a data governance policy and i was told no but when i arrived at my desk the next day there were two binders of policy that were all data policies none of them were data governance policies but they had policies for a lot of different aspects of data management and the management of data related assets metadata is one of those data related assets that we know that if we're going to improve the visibility and the usage and the quality and value of the data in your organization we need information about that data and that's the metadata and the stewards themselves the people that are being held formally accountable for defining producing and using the data there data related asset as well so we want to make certain that we incorporate things like these topics into our policy governance to make certain that the policy is covering those things that are going to be necessary for us to be successful at governing data across the organization in this slide i kind of highlighted or should i say grayed out my definition of stewardship with formalization of accountability of the management of data and data related assets well we want to make sure that we're formalizing the the data policy for the organization make sure that it's not just shelfware it's not just a document we want to formalize accountability from all three of these for the definition the production and the usage of the policy itself um as i said before and i say often that the data won't govern itself the information the metadata won't govern itself well the policy is not going to define produce and use itself so look to see if there is a process in your organization for how policies get defined and then look and see what data policies you already have to determine whether or not a data governance policy per se is necessary within your organization and oftentimes the the steps that we go through to get a policy approved is it starts somewhere sometimes especially a data governance policy may start or begin in the data governance office or with the data governance administrator or data governance administration or they are data governance lead or data governance manager whatever you call the people that have the responsibility for your governance program they may be the ones that think that the data governance policy is needs to be in place but oftentimes after they develop that policy they'll take it to the sponsor or the person that's ultimately accountable for making certain that data governance happens you know once they've gotten feedback from the sponsor then they update the policy and they can take it to the data governance council i've given webinars often on data on data governance roles and responsibilities and the council is at the strategic level of the organization and the operating model that i share but oftentimes it'll go from the office to the sponsor to the council before it ever makes it to the executive committee or the senior leadership of your organization so there probably is already a process in place for getting policies approved within your organization we want to use the policy to strengthen data governance so again as i said before it strengthened senior leadership support sponsorship and understanding because they are not going to sponsor it unless they are our support it or sign it or approve it unless they are asked to do so and oftentimes it's great to have a policy so you can lean back on it and people ask well who really cares about data governance or senior leadership does to the point that they put this policy in place they signed off on this policy and if they ask the question who says that data governance is necessary well obviously if we have policy associated with data governance it's deemed as being necessary to the organization so uh auditors and the examiners may say that it's necessary so it may not be up to you whether you're a policy driven or not policy driven organization you know that your auditors and examiners may say that you're required to have a policy a client of mine recently their examiner said they needed an information governance policy associated with records management or the management of unstructured data and typically policy is reserved for something that is important enough to require that level of formality in your organization and so use the idea that other governance policy is important already for the need to govern data and protect data but use those policies to demonstrate the value and the need that should come from a data governance policy if you decide to go that route so what does data governance look like without policy well as i said before i will talk about best practices a lot and if i told you that 99 percent of my clients use senior leadership must support sponsor and understand data governance as their number one best practice i'd be lying to you because it's more like a hundred percent of the clients or the organizations that i work with now that may not be everybody uh not yet but it may be that you know that all of these folks find it's important enough and they recognize that we'll be at risk if we don't achieve the formal governance of the data if we don't have the senior leadership support sponsorship and understanding so it is uh it's important that you have something to point back at and say well we do have a policy it is felt like it is important enough from the highest levels of the organization so data governance does not require that you have a data governance policy or a policy that's called data governance policy you probably already have governance through existing policies it doesn't hurt your organization though to have a data governance policy or something that defines the four core principles that i'm going to share with you in a minute so let's get into that right now let's talk about the makeup of a governance policy and like i said there was an earlier webinar maybe a couple over the years that i've been doing these with data diversity they're kind of outlined the steps that are necessary and what goes into a good data governance policy you know so there's different components i'll talk about the core principles i want to spend a little bit of time on that i share a one pager that is a that is an example of a policy with abc company and that's a generic company but it's just an example of some of the statements that you might include within a data governance policy and then we'll talk again briefly about gaining approval for the policy so what are the different components of a data governance policy or really any policy for that matter in your organization and as i said i laid that out in a real world data governance webinar with data diversity back in april of 2017 the the webinars around policy seem to be attended very well because a lot of organizations have that question as to whether or not policies are necessary so oftentimes in the policy there's an introduction of why there is a policy what we're covering in the policy what are the consequences of not following the policy then there's a policy statement and in the sample i'm going to share with you it has a sample policy statement and then the principles i'm going to spend a few minutes on the four guiding principles that i think need to be included within a data governance policy and one of the end results are the different dimensions of data quality within your organization if we're going to achieve accurate timely consistent complete relevant data for our organization that's how we measure the data but we can also measure that in terms of policy are they helping us to achieve improvements in any of these dimensions of quality so this may be the most important slide this in the next couple slides may be the most important slide of the slide deck if you're thinking about creating a data governance policy and they're the guiding principles and i have a graphic i'm going to show you that i've used and i've shared over the years of where these kind of fit in the whole picture of data governance policy but the first one is one that we've heard all too often maybe not too often maybe not often enough that data is an asset to the organization and it needs to be recognized and it needs to be valued in such a way and it needs to be managed as if data is a strategic enterprise asset and that could be data it could be information which i uh say information is data plus the metadata the context turns it into information and the metadata itself is a valuable asset to the organization the accountability for the data is clearly defined and enforced that's necessary anybody who's developed the data governance program knows that the roles and responsibilities and who's accountable for what is is very necessary as the backbone of your program and it needs to be clearly defined and enforced across your organization that is what is everything is related to those roles and responsibilities and the whole concept of stewardship and formalizing accountability for the management of data we need to understand who's accountable i had some organizations tell me we have no accountability well the chances are that you wouldn't be a 20 year old company with 50 locations around the world if you're not if accountability isn't really something that's important to your organization data is managed to follow internal and external rules and regulations and that's a that's a no-brainer we know that there's rules and we're not they're not coming to us and asking us if we want to follow follow these rules they're telling us that we have to figure out the best way within our organization to follow these rules data quality is managed and defined consistently across the data life cycle that's the fourth um guiding principle so i'm just going to go through these real quickly again data is recognized as a valued in strategic enterprise asset that's the first guiding principle the second one is that accountability for the data is clearly defined meaning that somebody needs to be responsible for it we need people to recognize themselves as being accountable and we can certainly as data governance practitioners help them to do that that the data is managed to follow the rules again the no-brainer of the group is we know they're not asking us they're telling us we need to follow these rules we need to determine what the best way is to do that and then that the data quality is defined and managed consistently across the data life cycle so those four guiding principles and you may have seen this diagram before maybe in this format maybe in a different format where they're just right down the middle of the diagram as you can see the guiding principles right smack dab in the middle of the diagram and then they all have like these bubbles that come out of them that are like how do we put this into common people's language um you know instead of saying it's valued as strategic asset we're moving it from being my data to our data or the data has to become everybody's responsibility or we want to get things right the first time so they're just kind of talking points around each of those principles but your vision statement and your policy guideline statement basically needs to be broken down into these principles and they need to be measured in such a way in the organization that you can demonstrate value to the organization and again the dimensions that I mentioned quickly you know accessibility accuracy completeness consistency all of those things are really important when it comes to trusting and knowing the the guidelines for how we improve the quality and the value of the data in the organization so here is a sample data governance policy abc the art bark brewery company I don't know what abc stands for but the company's policy is designed to manage the creation transformation and usage of data and related information owned by or in care of the company the policy applies to everybody in the organization employees contractors temper temps consultants authorized agents it is the policy of the company to require that all data defined produced and or used by the organization in care of the company must be governed as a resource that's going to help our company to move forward all users must maintain the quality and the integrity of the data resources violations of this policy I said we need to put consequences in there as well so violations of the policy may be considered a serious breach of trust which can result in disciplinary action there have to be consequences in place for not following policy so I spoke a little bit before about gaining the approval for the data governance policy first we need to recognize that there's a need for a policy utilize whatever organizational construct there is for policy and don't create something that looks you know very unique but follow the marching orders that your organization has set up for policy draft the policy validate the policy with the sponsor presented to the strategic level the council then take once it's you get them to concur that this is the right thing to do take it to the executive level for signature that's going to demonstrate they're not going to sign it if they don't believe it they don't trust it they don't think that it is necessary for your organization so assigned policy is a way to be able to demonstrate support sponsorship and understanding of data of data and the need for data policy in the organization so we want to make certain that as part of this after it's been approved that we socialize the policy to the organization through orientation of people to the policy through onboarding when they're being asked to actually do something that that has to something new with the policy that's been defined and then enforcing the policy with consequences I keep saying that and it's really important that it's not really a policy that people are going to follow unless they understand that there are consequences for that policy so the last subject that I want to touch upon here is is having a policy to manage data policies data data or should say policy management policy that sounds like meta policy like metadata but some of the drivers in the policy purposes expanding policies through policy comparing policies to other things like guidelines and standards and what are some of the alternatives that we have when we're looking at policy why do we put policy in place to demonstrate senior leadership backing to increase the awareness of people across the organization as to the importance of governing data that would be the reason why we would put a data governance policy into place to formalize authority to who are the decision makers within the organization how do we take something and get a decision made and also to formalize accountability again back to my definition of what a data steward does is we formalize accountability for the way people define producing used data across the organization and then I keep talking about the consequences for lack of adherence that's really a driver as well which we want people to utilize data to protect data the way it needs to be protected then we need to make certain that there are consequences for not following the rules and it also helps to document the importance of data governance to the organization so formalize your method for your policy requirements and development formalize your method for how you're going to structure and build your policy formalize how you're going to take these policies through the organizations you get them approved and make sure that you keep record of the policy and that you communicate them and take them off the shelf and make them necessary for people to know they don't necessarily need to remember them word for word they need to know that you have policies associated with this that and the other there's certainly policy around entry into a building yeah remember the days when people used to do that well that's going to happen again so you know what what's the access policy look like even to the organization we need to make certain that people know what policies are available and the data governance and data management policies might be included in those or they're already being included people just need to know that they're existing and what it means to them and you also need a formal process for reviewing and maintaining the policies like I said business rules could change regulations could change business could change so you want to make certain that you're doing all sorts of those things so comparison now let's just real quickly compare the policy through the guidelines to standards to agreements a policy is an approved course of action a guideline is a provided instruction of behavior oftentimes referred to as the guide rails you know we've got to do things we've got to follow these rules at least but there are more guidelines they're not signed off one I don't know organizations that have their guidelines signed off at the senior leadership level standards are requirements that need to be followed and so it's different than policy it starts getting into the nuts and bolts of there are data quality standards there's data definition standards what do we need to include within a definition and then arrangements are even less formalized they're just kind of put in place to formalize people's understanding of what's necessary within the organization so those things are all different and then there's alternatives to policy as well there's rules there's strategies plans procedures directives agreements and I've provided definitions for each of these things you know the rules depend on the formality and the requirements to follow the strategies typically come from a higher part of the organization it's kind of a direction and a vision for the organization is included in your strategy your plans are really how you're going to carry out the actions that are necessary to enforce the policy across the organization procedures include the steps to complete those actions the directives those typically come from senior leadership stating that that this is what we need to do this is what we need to follow and the agreements are basically getting people to commit to following the rules you've heard of you know shared shared or SLA or service level agreements within organizations when they sign up on those they're agreeing they're committing to follow certain rules that are defined for the organization so the last slide before I kind of do a quick summary and then turn it back over to Shannon just talks about policy management policy so we need a formal method for how policies come into existence and you know what we need to govern that whole process of creating the policies we need a plan for defining and producing and using the policies across the organization a plan for who creates the policies how we're going to get them approved and the term that I wouldn't be surprised that if someday would somebody would start talking about it is metapolicy what is the policy of our organization around the development and the delivery and the the consequences that are are required if people don't follow policies so in this 50 minutes I shared with you a bunch of information about data governance and policy management I talked about the relationship between governance and policy and provided my definition of each of those I just wanted to recognize that policies are already in place in your organization talk about using the policies to help bolster your data governance program to demonstrate your senior leadership support sponsorship and understanding we spent a little bit of time talking about the makeup of the data governance policy and specifically those four guidelines that are often included within or or are similar to the principles that are provided within a data governance policy and then the last thing we talked about was having a policy itself to manage policies to manage data policies across the organization so thank you for listening to this session and with that I will tell you that I will be in the dataversity community after this webinar is over but I wanted to turn it over to Shannon or back over to Shannon to see if we have any questions today. Bob thank you so much and just answer the most commonly asked questions just a reminder I will send the above email by end of day Monday for this webinar with links to the slides recording and anything else requested throughout so diving in here Bob to the questions I have two questions one how do you apply this data governance policy rationale for national data strategies and two could you please further develop your point on the role of auditors in the context of data policies well so that's the first part of the question again and then I'll answer that in the auditors. Sure how do you apply this data governance policy rationale for national data strategies? Well national data strategies and even global data strategies may start with the fact that we've created this this strategy because we have a policy that says that we need to have it in place so I think it just kind of further cements the the whole idea that that this is something that is recognized and that is important across the organization and so again the second part of the question was is could you please further develop your point on the role of auditors in the okay and so oftentimes you know I just used as the example during the webinar that the examiners for a recent client of mine said one of the things that was the result of their examination report was that there was no policy associated with the protection of classified information or the protection of sensitive information so they told this client you need to have an information governance policy in place so that there is steps are documented and required and people know them it's been communicated to people that we need to do this so the examiners and the auditors actually can help to define well what do we think the external auditors are going to do I start by working with the internal auditors say well what do we anticipate some of the questions of the external auditors to be and let's make certain that we put a policy in place that will help us as an organization to address what the examiners and what the auditors are looking for do you think C level support would be necessary senior leadership tend to be very territorial and sway to different directions when things get controversial I not only think it's very important but I would say it's critical that if you don't have senior leadership supporting sponsoring and understanding what you're doing that you are going to become at risk very early on in the development of your program in the enforcement of your program and I mentioned throughout the webinar that one of the ways to for them to demonstrate their support sponsorship and understanding is for them to sign the policy and you're right some of the leadership can be territorial and can be specifically looking out for their own needs but that's not typically at the highest level of the organization you know they're going to be saying well what are we doing that's in the best the best interest of our constituents across the entire organization um you at least I've rarely experienced that organizations have policies only set up for certain divisions within the organization if it's important enough to be a policy it's important enough to be a policy for the entire organization and just kind of to add on to that but does data governance enforce the policy that's a big barrier for us it has to be senior leadership and accountability well they can help to enforce it but they're not the data police so I don't like that term data governance and data police that's not what their responsibility is their responsibility is to make certain that the things are in place the components of successful governance are in place so you know I'm not sure that data governance itself is going to police the policy but they can report occurrences of the policy not being followed so they're not there to to come by and to take you away from your desk and put you into data jail in your organization they're not policing it that way but they are monitoring and the data governance group you know if they're working significantly with other parts of the organization they know what behavior is taking place and if there's behavior that goes against the policy they need to bring that to the attention of senior leadership they're not the ones that are going to reprimand they're not going to be the ones that provide the consequences or I'm not talking about the data governance office or the data governance team but that has to come from the top if people can get away with breaking policy without any consequences people are going to get to the point where they don't feel as though the policy even means anything and therefore it's kind of a it's not even necessary within the organization what is the relationship between policies and standards do all policies need accompanying standards so that they know clearly how to follow the policies well I we say they do need information about how to follow the policies but the standards that probably a lot of people on this call are talking about are the data standards data quality standards I don't know if they're called standards within policies I'm not that much of a a policy developer to know that in all sorts of policies that they have standards but they might have guidelines as to what people need to do to follow the state to follow the policy itself I don't know if you have a data quality policy it might include the standards around those dimensions that I spoke about earlier but they're not you know you're not necessarily going to have standards in every policy in fact a lot of the policies and the one that I shared did not have a section in it that was called standards again look to see what already exists in your organization as a construct for policies and if it does it does have standards then by all means include those within your policy and Bob can the DMBock or the data management body of knowledge we'll be used to guide the development of policies and standards so I don't know the DMBock that well I know of it I have a copy of it and then it's not something that I've read from to back I don't know if there's a section in there around data policy but I would think that you could get a lot of disciplines and a lot of ideals from the DMBock it would be very beneficial it's very much a great way to start a course of study around data management it has a lot of different disciplines in place so I would think yeah you would benefit from incorporating some of the ideas that come from DMBock into your policy but I'm not sure that they I don't I could be wrong somebody could correct us in the chat here if they know that there is a section in the DMBock about the about the development of a policy I would think that there's a fairly good chance that there is all right well I think I can slip in one more question here and if you have additional questions keep them coming in I will send them over to Bob who will write up answers for you to be included in the follow-up email so what are the most important policies to begin with in a new data governance program most important policies well you can always point in your information security policy because data protection and data classification and making certain that sensitive data that is data that's rated highly confidential or confidential even sensitive you know basically any data that's not open data or public data that there's rules associated with it there is oftentimes data retention policy within an organization how long do we need to save to save the data to keep the data around for legal reasons so data retention would be another one data access would be one as well data privacy policy a lot of those have to do with similar items but that would be where I would look first I don't know of as many organizations that have data quality policy as many of those that have that type of policy as compared to those that have data protection and data privacy policy so I'd start with information security privacy you know those types of regulatory controls and then work your way out from there I love it well Bob thank you so much for another fantastic presentation appreciate it as always and thanks to all of our attendees for being so engaged in everything we do very much appreciated and again just a reminder I will send a follow-up email by end of day Monday for this webinar with links to the slides and links to the recording hope you all have a great day and stay safe out there Bob thanks so much thank you share thanks everybody thanks all


Hey everyone, this is Brian, and in this video we are going to make a complete application using all the knowledge we've gained in this series so far. Now this application is going to be a very simple, I have to stress the term simple, inventory application. At its core it's going to be a QMAP that we're going to save to a file and load back. We're using a QMAP because we want a key value pair like an item number or an item name with a quantity attached to it. And of course we're going to have some extra code in there so we can add or remove items from that inventory. I'm going to do this in this series from time to time because our YouTube series is, well I like it but each video is very self-contained, meaning one video covers say files, the next video covers say Q-list, they're very self-contained and these specific videos where we build an application takes all these parts and kind of puts them all into one so you see how all the pieces fit together to make a complete app. So I am going to do this from time to time. I haven't really decided on like X number of videos so it's not going to be like every tenth video or anything like that but drop a comment below, let me know how often you want to see these. Now because we're still in newbie land in this series this is going to be a command line application but yes as this series progresses we're going to do QML and widget applications as well.  This topic would be touched on in the Qt6 core for beginners with C++ and the Qt6 core intermediate with C++. This video also is going to hyper focus on Qt data stream which is how we put streams of data into an IO device specifically a file. Let's dive in and take a look. Okay let's start at the very beginning I don't even have Qt Creator open I'm going to walk through this whole thing step by step so this is Linux Mint but again Qt runs anywhere so you may be on Windows Mac whatever this code will run just perfectly fine on all of those platforms. So we're going to go to programming we're going to open up Qt Creator and then from here I'm going to say new and we're going to choose console application. Now I need to give it a name name this whatever you want but this is going to be the name of your executable for my purposes because I want this to follow a specific standard out on GitHub that's right the code for this will be out on GitHub I'm going to say Qt6 and we're going to say episode and this is episode 20 put it in whatever folder you want and hit next we're going to use the CMake build system I'm not going to have a translation file kit selection I only have Qt6 in here if you have an older version like Qt5 you may run into some little issues some little differences between five and six remember this series is covering Qt6. It next not going to use any project management and here we go we just have a blank application now I'm going to go ahead and I'm going to right click our project and I'm going to add new and I want to make a class and let's go ahead and call this inventory this is really a design decision and unfortunately when it comes to design decisions there's a million different ways you can do this I'm putting my business logic into a class and I'm going to make this a Q object that way I can call it via signals and slots if I decide to make this application more complex in the future need to make sure this includes Q object and add the Q underscore object macro that officially makes it a Q object we have our header and our source hit next and then just hit finish this little guy is annoying Q if you're watching please pretty please make this go into CMakeList.txt automatically but for now inventory see gotta type this out hit save make sure it adds the header and the source file and we are good to go now we've got our class here but it's well just a class there's really nothing in here it's just a blank class we need to flesh this code out and I'm going to break this up in a specific sections but this section right here we're going to talk about the class design so the first thing I want is some includes and from time to time I do copy and paste I have some notes off the screen just to save a smidge of time typing we're going to use Q map Q debug so we can see some output on the screen Q files so we can work with you guessed it a file and then Q data stream this is something we haven't really touched on in this series but did cover in the Udemy courses so what is Q data stream if you highlight it hit F1 you can see the integrated help it's a class that provides serialization of binary data to an IO device an IO device of course would be a file yes you could also do this to like a socket or something like that it's really just that cool once you get more advanced but at its core a data stream will serialize the data meaning it's going to take it and turn it into a language or code that the machine understands but this is not human readable it's much more efficient to store and to retrieve that's why we're going to serialize it if you want more help just highlight Q data stream hit F1 and read down through the file but we're going to cover some of the basics of this class and there's a lot to it so I stress the basics from here we need to make a couple other design decisions as well I'm going to have some functions and I'm going to just do some copy and paste we're going to say add remove and list our inventory system we want to be able to add you guessed it some item with a quantity and remove some item with a quantity and then list basically we want to know what we have in our inventory from there I'm going to make a minor design decision that at this point in the series may not make a whole lot of sense I'm gonna say public slots so notice we already have signals and now we have slots so signals and slots we've touched on these but basically a signal lets the outside world know hey something happened some event happened it's emitting a signal a slot would consume that or take some sort of action this is event driven programming so cause and effect signal slot so when a signal is emitted you can connect to a slot and have that slot be called automatically from here I'm going to paste in save and load so the first question I'm often asked is what what what's the difference between a slot and a function virtually not to be brutally honest with you you can call a slot like a function but in order to use cute's built-in event-driven system called signals and slots you have to have a slot and this is where mock the meta object compiler comes in basically because we have this Q object macro when we compile it's going to see this and it's going to scanner class and go oh you have some slots and it's going to convert these functions into magical functions called slots which does all that event-driven programming in the background for us if it sounds confusing because it's actually pretty confusing under the hood but let's go ahead and flush this out and I'll show you what I mean here I'm also going to have a private variable called m underscore items which is going to be a Q map with a key value pair of Q string and int so there's our item name and our quantity from here we're just going to right click refractor and then add definition I'm gonna do this for each one of these they may be asking yourself why as I'm doing this why did we make those two slots but the others are just normal functions again it's just simply a design decision when you get a little bit more advanced you will literally take these and just copy and paste them under slots that way all of them are slots because you'll see the power of slots it's how you would get one Q object to automagically that's a word now automatically talk to another Q object by a signals and slots but you see in the implementation it's just normal functions so we can call save and load like normal functions or we have the options of using event driven programming signals and slots I'm gonna go ahead and save and I'm going to build we got a good build so we are ready to start fleshing out code but before we do let's jump out to the hard disk I want to talk about mock super quick we're gonna go to cute this is our build folder and here is our actual binary on windows you're gonna see like dot exe and you see we have some stuff here now let's look at this auto gen oh la la look at that mock meta object compiler so this is what's going on here is it's actually going out and generating this code and you see here is our mock underscore inventory our class was named inventory and our slots are save and load and if we crack this bad boy open you'll see save and load so that's what mocks doing in the background is it's taking any function that we define as a slot and it's generating this mock code for us so that as we make more complex applications we can connect using signals and slots I wanted to really cover that in this video even though we're not gonna use signals and slots very heavily because I'm gonna get those questions later on let's start filling in some code I'm gonna break this into two sections here so we're gonna do this code first and then in the next little section we're gonna do this code why because this is the inventory code this is really our internal business logic and this is our serialization code I want to really in your mind split those into two separate pieces anyways we can just switch to header and source and right let's go here so first thing I want to do is make a design choice again there's a billion different ways to really build this application but in the constructor I want this to automatically load our information which we're not gonna fill out just yet we're gonna load and this is going to load any file that we previously saved now notice we're calling this like a normal function that's what I mean by slots really are normal functions with mock meta object compile wrapped around them now from here we need to start flushing in some code so when we add an item you'll say if I'm underscore items my horrible typing included it no extra charge it's gonna say contains I've had this horrible I shouldn't say or is a great gaming keyboard when I bought it but I have to beat the hell out of this thing on different video games so I probably need to invest in a keyboard at some point and my mouse is the same way so once in a while you'll see me actually fight with the mouse I think it's kind of busted to be honest with you all right so what we're doing here is we're saying if our map M items contains the name or the key then we're gonna go ahead and say go ahead and take that item and add the quantity to it however if we don't already have that in our inventory we're gonna insert it we're gonna say m dot items dot insert and we're gonna insert a key value pair in this case our name and our quantity now somebody out there is going to say well if you call insert it will automatically do it for you the problem is it will overwrite the quantity instead of add to it so like if you go here F1 it says inserts a new item if there's already an item with the key that items value is replaced you gotta be a little careful there right now that we've got this we want to do the exact opposite which is removed so I'm gonna grab this copy paste and we're gonna say if it contains our key here then we want to do the opposite we want to reduce it but we've got a problem what if they go beyond zero so now I'm gonna say if m items dot value we want to get that value is less than zero and we're just gonna go ahead and remove it and this will actually take it right out of our map all together now I'm a big fan of letting the user know what's going on so I'm just going to say item removed now as you're watching me fumble around with this keyboard let me know if you prefer watching me type or if you prefer watching me do the old copy and paste like I'm doing there and I know some instructors actually have programs which will automatically type out all the code for you I don't use those I think they're a bit cheesy but again you wouldn't be watching me make a lot of mistakes so a quick recap we're adding and we are removing if we're removing an item and we go below zero we're just going to remove it right out of the map so we don't have negative numbers in our inventory alright to wrap this little section up we're gonna go ahead and list so I'm gonna say Q info items and we want to know how many items we have some say M items dot size there is a count which will return the same number but you can also count for specific items in there so we're just going to use size to avoid any confusion I'm gonna say for each I'm gonna say Q string key in M items dot keys and I'm pretty sure we covered this in our YouTube series I know I definitely beat this up in the Udemy courses but keys is just going to return a Q list of Q strings or whatever our key is remember we defined our Q map key value pair as a Q string for the key and a value as an int so we're just gonna say for each Q string key in the list of keys and then I just really want to print this out on the screen just so the user can see what we have an inventory it's actually pretty simple when you look at it in this terms so the major takeaway here is there's different ways of doing this we could have done the C++ way we could use for each we could have done a for loop using you know begin and end doesn't really matter as long as you get the result you want the end user simply doesn't care let's work on our serialization code so we've got two functions save and load save and load are pretty self-descriptive save is going to save to a file and load is going to load it back so let's save it first I'm gonna say Q file and let's call this file and we need a name inventory dot txt we can name this dot dat or whatever we wanted I'm just gonna name it dot txt so we can just crack it open very quickly in a notepad and see what it looks like I'm gonna say if and we're gonna go ahead and open this file file is not openable is that a word openable say Q IO device we want to open this in right only mode if we're not able to do this well we have some problems and we need to tell the user hey and no bueno couldn't do it so we're gonna say Q critical could not save the file and then we want to get the error string out of the Q file object that way we can tell the user hey something bad happened go deal with it and then we're just gonna hit the eject button on this and jump right out of here now comes the interesting bit this is something I don't think we've really covered a whole lot we're gonna say Q data stream and we're going to give it our file so what this does is it opens a file but now we are going to put or serialize some data into that file and we're gonna use Q data stream to stream the data and let it encode it and decode it for us but before we get to that point we need to tell it how so I'm gonna say stream set version and this is critical a lot of people and I'm guilty of it too we'll skip this step but we're going to explicitly say we want to work with cute version he knows it goes all the way back to 1.0 we want to set a version and say this is the version we're going to work with that way if somebody hands us a file that was encoded with an older version or a future version we know hey it's going to have a different encoding under the hood and we're going to check for that in the load section here but anyways what I want to do first is I want to write the number of items to the stream so I'm gonna say int Len equals M items and we want to get that size so I just want to know the number of items in that map and then writing it is extremely simple watch this that's it we're just going to pump it out to the stream and let the stream deal with all of the IO all the encoding it can just do what it does best and then I'm going to just give the user some sort of feedback that we're doing something number of items to save I'm gonna do a for each some people love for each some people hate it I've just gotten really used to doing it over the years but there are newer snazier ways of C++ to do it and we're gonna say M items that keys bang so really all we're gonna do is we're gonna go through each and every single little key here and we are going to save it so okay bang and then let's go ahead and say stream and we want to pump that key out the stream stream and now we want the values we're gonna just take the key value pair and put them in there but notice the order we're doing this in so we're putting the number of items then we're doing key value key value we're just gonna loop through that when we load this back the order in which we saved we have to read it in the same order we're gonna get corrupt data so be very mindful of that you'll see what I mean here in just a second so we're gonna say file close and then Q info I'm really envious of some of those instructors who will like have a program type it all out for me actually look into that I know I know I've asked that before and everybody's like no don't do that we like watching you type because we like watching you suffer anyways alright so I'm just gonna copy all of that code quick recap what we're doing here is we're opening a file called inventory.txt you can 100% put that somewhere else we're gonna say if we're unable to open the file in write only mode then we want to let the user know we're gonna make a data stream set the version and then we're gonna pump our data out and we're doing this in a specific order the number of items and then key value key value key value over and over until we're done and then we're gonna close the file note in the close it will also flush so any data that's sitting here pending in the buffer is going to get flushed down to the desk flush just means it's going to force it to write down to the desk now load loads a little interesting it does the exact opposite so I'm going to paste this in here we're gonna say if file open in read only mode failed then it could not open the file this is where I always make mistakes cuz I get it all screwed up let's back up here real quick I want to actually check to see if the file exists some say if not file that exists then we're going to and I'm gonna change this to a warning some people are gonna say what is the difference really between like a Q info key debug key warning these are levels of logging in case you kind of skip those we really really beat those up in the Udemy courses but you can actually build your own logging system and then set levels and categories and all this other cool stuff but we're just going to say file does not exist that way the end user knows hey we got no inventory there's nothing to work with here hit the eject button if we're unable to open it and read only mode hit the eject button now comes to the meat of it here in our save code we did a set version so I'm going to grab this version and we're gonna check for this we're gonna say if get the mouse out of there yeah their mouse alright so we're gonna say if and we want to know the stream version is not equal to the version we're using then we've got problems and we're going to simply say Q critical now in a real world application you would try to you know match the version then use the function that uses that specific version and so on and so on or say hey here's our data migration wizard that our company has built that will migrate your data to the newest version but instead we're gonna say wrong data stream version because there is one and only one in the land of our inventory app then we're going to just say file closed because at this point we have an open file and we're going to hit the eject button some of you out there are probably going to be really really good at reading the documentation we really don't even need this because as soon as the Q file goes out of scope it's going to close itself automatically I like to make sure that I'm closing it because I want to be in control alright from here we have an inventory system we need to clean that out I'm say M items clear and that's just going to unapologetically wipe anything we already have in memory out of there and then we can start loading it so we want to know the maximum and then we're going to start working with some stuff so I'm gonna say stream and we're going to read in that max you may be wondering wait a minute how do you know what to do well if you go back to our save code the first thing we did was we saved the number of items so we're simply just reading that back and then we're going to do the same for each key value pair so we are saying int max we're reading the number of items we're going to read back in and then we can say Q info number of items to load and then I'm going to just wipe this stuff out so we don't get that confused actually I'm gonna say loaded okay so now comes well you may be thinking this is the confusing bit but this couldn't be any simpler I'm just gonna say four and I equals zero I is less than our max I plus plus so we're just gonna increment our eye now we're gonna work with a key value pair so I'm gonna say we want a Q string key and it Qt y and we're gonna read those in so I'm gonna say stream E and then stream Qt y again you got to read these back in the order you saved them remember in our save code we saved the key then we save the quantity key quantity so of course down here we have to do key quantity key quantity now that we have those loaded up we're gonna say m items insert and we're going to just simply add our new key value pair from here files gonna close and we're gonna say we have fully loaded that out to the end user so they're aware of what's going on quick double check before we move on let's make sure we've fleshed out every single little function and remember our constructor is gonna call that load function we just fleshed out so as soon as the class is constructed it's gonna go out there and try to load that file if we previously saved it okay now comes the fun part we need to flesh out our main function this is actually the bulk of our application here so I'm going to just copy and paste some stuff this is our simple inventory application we've made a class we filled in the code now we need to monitor the user input and we're going to use the Q text stream the difference between Q text stream and Q data stream is that data stream is a non readable where text stream is human readable so this is what we're gonna use to get the end users commands and we're of course going to use our inventory class that we just built I'm going to judiciously copy and paste code to save some time we're gonna create an instance of our inventory class we're gonna tell the user because users are dumb these are the things you're allowed to do notice some of the commands like add remove you have multiple items add space item space item remove same thing but others are just one so we're gonna have to split those up just be aware of that so I'm gonna say whoops not qtex format qtex stream this is why I copy and paste things because I can't type and talk at the same time darn it all right so we're gonna say STD in and what this is going to do is it's going to wrap the standard input with a qtex stream so we can use that functionality and this is where we need to make a design decision I'm going to loop forever but this is really poor practice we're just simply not there in our little tutorials there's a better way of doing this that we're gonna cover when we get a little bit more advanced some say Q info and we are going to tell the user because users are dumb enter a command probably not nice for me to say users are dumb but you know what I mean you all have that one user we got to like walk them through every little thing now the big problem here is because this is gonna loop forever we've got to be able to know when to stop looping so let's go ahead and say Q string let's put this line and we're gonna say stream dot read line so we're gonna take our text stream and we're gonna read a single line now from here we need to take that line and break it up because some of these will have multiple parts and we're gonna use what's called split some say Q string list I'm say line split and we're gonna split on the space because that's the format we chose again design decision we could have split on a comma or whatever we wanted to do now that we've gotten this far what we're gonna do is we're going to say all right I want to know what the end user actually just entered some say Q string man equals list at and I want the first position dot to upper and we have a subtle bug in our program I don't know if you caught it but we're assuming that there's even an item in here so we should probably back up here and say if list size is less than or equal to zero then we're gonna go ahead and break out of here that way if something weird happened the entered some gibberish or our program freaked out remember these are cross platforms so they're not gonna behave exactly the same on every single platform we just want to make sure that we have some sort of list to work with our command is going to be the first item at the zero position to upper and from here we can do some very very cool things like if man equals quit and we can say either a dot quit you're wondering what a is it's our actual Q core application and you see we're entering an event loop here so we can either quit or we can say a dot exit and specify an exit code either way we're gonna break out of here spoiler this may actually misbehave on some systems and we get more advanced we'll talk about how to fix that but just beware it's going to behave a little bit differently on everybody's systems I've seen it behave mostly the way you expect but sometimes it does just do weird stuff but now it's really pretty basic actually I mean we can just save just a smidge of time and say okay we're going to save the command is list we're gonna list if it's load we're gonna love save we're gonna save now if you copy and paste you get this like weird formatting like I would I got you can just control a and then you can go to refractor and then I'm sorry not refractor auto indent selection or control I and it will fix it for you absolutely love that feature because I copy and paste a lot in my videos but you notice what we're not doing yet we're not adding or removing and these are the tricky ones because in our little protocol they have multiple parts add or move so rather than treat each one separately what we're going to do is check both homes say if and those add or man and this is where horrible spelling like I have will come back and haunt you and that's why in more advanced programs will actually use an enum instead of using a hard coded string again design decision based off of the knowledge we have so far if we're using these commands we know we need at least three things right we need a command a key and a value so I want to make sure we have those list size is less than we said we need three things then we're gonna tell the user hey do not pass go instead go directly to jail do not collect $200 that's a a board game in case you don't know what that is sex approved fun game alright so we're gonna say command add or remove if we don't have enough items we're just gonna say not enough info and we're going to continue meaning we're gonna restart this whole loop all over again and ask them for another man if we've gotten this far we're gonna go ahead and get that information some say Q string name list at we're gonna go ahead and get the first item now remember first is not first actually the second because it's a zero based index the zero position is actually our command the one position is our key now we're gonna go ahead and we're gonna say bull okay it's QTY the reason why we're gonna use a bull is we're going to convert from a string to an integer but we need to make sure it actually works the way we expect some say QTY equals list dot at and we're gonna get at the second position I'm gonna convert this to int and we're gonna give it our bull so we're gonna say if it doesn't work I want to know so if this is true it worked if it's false it didn't work and then we can just simply check that by doing something like this and again horrible formatting we're gonna say if it's not okay meaning if it's false in valid quantity so here we're doing some conversion I get asked that quite a bit how do you convert from int to string and then once we have this again it is ridiculously simple we're gonna say if the commands add inventory add if the commands remove inventory remove now when we get to this point sometimes we have to save the users from themselves so this is another design decision I'm gonna automatically save this once we break out of that loop so I'm gonna say inventory dot save because you always get that one user that goes well I know I saved it and no they did not so we're just gonna save it for them just to be safe some people love that feature some people will hate it a production app you would be able to turn this on or off all right let's go ahead and give it a good build before we move on make sure I didn't goof anything up all right we've got a good build we can move on to testing okay that was a lot of code if you're still with me it's time to test this application let's go ahead and run it all right so it says file does not exist because we haven't created one yet here's our available commands because well let's face it and users need to be told what to do and enter a command so first things first let's go ahead and add something so I'm gonna say add cat and we want to add 22 cats because you know life's just not worth living if you don't have 22 cats now enter a command let's go ahead and add and we're going to add three dogs actually love dogs just as much as I love cats but all right we're gonna go ahead and list so we have two items cat and dog let's go ahead and quit now notice it says number of items to save to saving cat saving dog file saved file saved so that auto save actually worked for us but notice we didn't type the save command let's go out here and you see sure enough here's our inventory file let's crack this open this is what I mean by it serialized data this is not human readable and our little text editor this is a I'm on Linux Mint so this is a Zed it's struggling to try to figure out what this is because it's expecting text and this is not text it's binary encoded serialized data let's go back here close this and run it notice it says number of items loaded to so our auto load is now working let's go ahead and list this we've got 22 cats and three dogs sweet all right so let's go ahead and remove let's remove two dogs oh notice I misspelled that so are not enough info actually helped us here so we're gonna go ahead and try that again remove dog to now it actually remove we're gonna go ahead and list it we have one dog left and then cats 22 cats is a lot of cat food so let's go ahead and remove some cats here some say remove let's remove 19 cats now we have three cats one dog and we're gonna test that to see if we can actually remove and get into a negative value so I'm going to remove dog we're gonna remove 99 dogs item removed now if we go ahead and list notice it took the dog out so we don't have a negative number instead we just have cat and we can if we wanted to go ahead and save this manually and if we wanted to we can also load it so there you go very simple inventory using cute no application is complete without bugs so we've got a pretty nasty bug in this application let's go ahead and fix it before we finish this video first I'm wondering if you know what the bug is so take a moment pause and think about it okay enough pausing it's quit so if we look at the quit command we have a exit let's go ahead and look at exit tells the application to exit with a return code after this function is called the application leaves the main event loop and returns from the call to exec wait what the function returns a return code if the event loop is not running the function does nothing so there's the bug right there we're telling it to quit but we haven't even entered into our event loop yet so when we do this and we type the word quit notice this cursor is still blinking so this app is actually still running we didn't really quit our application oh alright and to prove that we can go into projects and then uncheck run in terminal say run and you see we're now running inside of cute creator under the application tab and we can type the word quit and it's still running ta-da so we're gonna force us to stop that is a bug we need to fix this so there's a couple different ways we could do this I'm gonna go back to run in terminal so we can see it in real time first thing I'm gonna do we really don't need the event loop and we can play around with a that exit but it really doesn't do anything so let's try this really and let's say quit now notice our application does stop because it says press return to close this window so now this did actually quit but it didn't really work the way you expect it because if the if we're not in an event loop the function does nothing so basically does nothing we break and then it goes all the way down here right so instead what we're going to do is we're going to return zero assuming that everything worked good because that's really all a exec does is it takes our q-core application interest in event loop and then once it exit it generates an integer so we're just gonna return zero we don't even need this to be brutally honest so I'm just gonna comment that out and let's save and run try that again ta-da so it's working as expected now now I don't want to mislead you there are probably tons of little bucks in this application because we're talking about this from a very simple beginners level we're only like 20 some odd videos into this series so expect as we continue these videos they're going to get more and more and more elaborate and complex and we'll probably have more difficult bugs to be brutally honest but we'll go through this journey together and figure it out if you're interested the source code will be out on get up I've been really bad about that but I will 100% upload the code right after I upload this video I hope you enjoyed this video you can find the source code out on github.com if you need additional help myself and thousands of other developers are hanging out in the void realms Facebook group this is a large group with lots of developers and we talk about everything technology related not just the technology that you just watched and if you want official training I do develop courses out on you to me.com this is official classroom style training if you go out there and the course you're looking for is just simply not there drop me a note I'm either working on it or I will actually develop it I will put a link down below for all three of those and as always help me help you smash that like and subscribe button the more popular these videos become the more I'll create and publish out on YouTube thank you for watching


So now we're going to look at a slightly different way to write out the equation for magnetic force using something called a cross product. So starting off with the equation we've already seen for the magnetic force on a charge moving through a magnetic field, we had F is equal to QVB sine theta. Where this last part of the equation was sort of our perpendicular multiplication between V and B that we looked at when we first derived this equation. Well, if we're going to use vector notation, there's actually a special way to write out this perpendicular multiplication called a vector cross product. If I write out the vector cross product using this vector notation, my force equation is going to look like this. Where now my vector force is equal to QV cross B. And this V cross B is this vector cross product between the velocity and the magnetic field. So I could say that the force is equal to the charge multiplied by the vector cross product between the velocity and the magnetic field. Note this is not just a multiplication sign. Don't say times here or just multiply. It's also not the X variable symbol either. This is a very particular symbol that represents a mathematical operation in vectors. If I were to take this equation and note that my vectors means I've got components for V and B in X, Y, and Z, then I could write out this cross product as a term by term equation using those components. And when I do that, it's actually quite a large equation here.  And then you've got these individual terms for I, J, and K, giving us the X, Y, and Z components of our force when we're all done. Because a vector cross product, unlike a vector dot product, you have a vector when you're done at the end. Now if you look carefully, you'll notice that for the X term, which is represented by I hat, I have Ys and Zs as the components that are involved here. Where I've got my J hat, which represents the Y component of the force, well I've got X and Zs here. And for my Z component of the force represented by the K hat term, that's where I've got X and Ys. So there's something in here about how each component depends on the other two components. Now another way to express this mathematically, which will make sense if you had linear algebra, is that I need to find the determinant of this matrix here, so I've got my I, Js, and Ks, and then my components in X, Y, and Z for the velocity in the magnetic field. Now if you haven't done determinants before, I'll give you some more instructions in another tutorial. But for right now, this is the other way of remembering what I'm doing when I'm doing the cross product. So that's our end of the introduction to the cross product. We'll still need additional practice to make sure you really understand what's going on.